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ABSTRACT 

This thesis introduces Concept Engineering, the result of an extensive 
collaborative research effort with product development professionals from 
member companies of the Center for Quality Management, as a complete 
decision support process for enhancing product concept development. Concept 
Engineering applies the principles and practices of Total Quality Management to 
develop customer-focused product concepts. The simultaneous introduction of 
Concept Engineering into product development organizations in three different 
companies created an opportunity for a comparative study of the product 
concept decision process. The comparative analysis is conducted using Inductive 
System Diagrams, a method introduced in this research, for systematic field- 
based hypothesis generation. Inductive System Diagrams combine aspects of 
grounded theory methods and system dynamics to develop and communicate 
substantive theories intimately tied to the data. The cross-company comparative 
analysis of product development teams, some using and others not using 
Concept Engineering, led to the generation of a dynamic hypotheses regarding 
the impact of a relative emphasis on TIME vs. MARKET orientation during the 
product concept decision process. It is proposed that a relative emphasis on 
TIME reduces concept development time but increases total product 
development time compared to a relative emphasis on MARKET orientation 
during product concept development. The MARKET orientation results in 
design objectives with higher clarity, credibility and stability. TIME orientation 
led to relatively lower design objective clarity and credibility resulting in product 
concept changes during downstream development activities thus increasing the 
total development time. 
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Preface 

The presentation of this thesis follows the progression of work done in this 
research effort. In the beginning, there was a desire to investigate the "customer 
focus" theme of Total Quality Management and it was realized that product 
development represented a high leverage point in which to conduct this work. A 
collaborative research effort with product development professionals from 
member companies of the Center for Quality Management led to the evolution of 
Concept Engineering as a complete decision support process for product concept 
development. This material is covered in Chapter 1 and Appendix A . 

The introduction of Concept Engineering to product development efforts 
within the participating companies created an opportunity for a inter-company 
comparative study. Additionally, as it was not feasible for the participating 
companies to implement a whole-sale conversion to Concept Engineering, it was 
possible to conduct an intra-company comparative analysis involving product 
development teams studying Concept Engineering and those that did not. The 
design of this research is addressed in Chapter 2. 

The research design led to an opportunity to conduct extensive, field- 
based participant observation of product development activities, in a 
comparative setting. As a result, it was possible to apply relevant theory 
generation methods from sociology to the product concept decision process. It 
was observed that a relative strength of the sociological methods was in the 
identification of key process variables. However, relative to System Dynamics 
methodologies for variable integration, the Sociological integration methods 
were weak. As a result, a marriage of the relative strengths of the two 
approaches was created in a process called Inductive System Diagrams. This 
process is described in Chapter 3. 
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In chapter 4, the hypotheses related to product concept development, 
developed through the Inductive System Diagram process, are presented in the 
context of current literature. Based on the generated hypotheses, management 
diagnostics for the product concept decision process are presented in Chapter 5. 
Chapter 6 describes potential next steps to continue the development of Concept 
Engineering, product concept decision theory and Inductive System Diagrams. 
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Chapter 1: Concept Engineering 



The total quality management (TQM) literature has two dominant 
themes: continuous process improvement and customer focus. The first 
theme, continuous process improvement, has a well-established set of 
methods and tools (7 Steps, 7 Statistical Tools, etc.) that are widely 
disseminated in practice and academia. (Feigenbaum 1951, Western Electric 
Co. 1956, Juran & Gryna 1970, Ishikawa 1976, Kume 1985, AT&T 1987, Scholtes 
1988, Montgomery 1991) In contrast, the customer focus theme, with the 
principal exception of Quality Function Deployment activities (Hauser & 
Clausing 1988, Akao 1990, Griffin & Hauser 1992) is not supported by a similar 
set of widely accepted tools and methods. However, an emphasis on 
attending to customer needs as a critical success factor in new product 
development has been consistently underscored by researchers in the quality 
literature (Shewhart 1938, Deming 1982, Ishikawa 1985, Juran 1988) and in the 
product development literature (for example; Rothwell et al. 1974, Cooper & 
Klienschmidt 1986, Clark & Fujimoto 1991). 

Motivation 

In Cooper and Klienschmidt’s (1986; p.76) study of 252 new product case 
histories in 123 firms "the weakest rated activities were the 'up front' or pre- 
development activities, namely initial screening, preliminary market 
assessment and detailed market study." Supporting this finding, many 
studies conclude there is potential benefit from research on the product 
development process, particularly the early activities (Rothwell, et al.. 1974, 
Cooper & Klienschmidt 1986, Clausing & Pugh 1991, NRC 91, Mahajan & 
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Wind 1992). Additionally, a recent National Research Council report: 
Improving Engineering Design: Designing for Competitive Advantage. 
estimates that 70% or more of product life cycle costs are determined during 
concept design, as illustrated in the figure below (NRC 1991; p.5). 
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The NRC report concludes the overall quality of engineering design in 
the United States is poor. Empirical studies of actual product development 
efforts confirm that critical activities are consistently omitted (Cooper & 
Klienschmidt 1986, Mahajan & Wind 1992). Additionally, the results of my 



14 



own field investigations in this area are consistent with this conclusion; I 
have heard CEOs of successful companies describe their product development 
process as "random walks with random results" and as a series of "blind 
alleys" (Burchill et al. 1992). In pursuit of this identified need. Concept 
Engineering was developed as a process for integrating customer driven 
requirements into design activities. 

Concept Engineering 

Concept Engineering is at one level a process for developing 
product /service concepts that strive to meet or exceed customer 
requirements; at another level it is a decision support process.^ This chapter 
outlines the evolution and essential features of the Concept Engineering 
process (a complete description is included as Appendix A) and then provides 
evidence that it is consistent with the requirements of complete decision 
support processes. A complete decision support process is defined as one that 
supports the decision maker in all phases of the problem solving process. 

Concept Engineering Evolution 

Concept Engineering had its genesis in the teachings of Dr. Shoji Shiba, 
a visiting professor at MIT, in the fall of 1990. Professor Shiba presented 
several Total Quality Management decision aides in the context of a quality 
deplo 5 onent case study. Coupling Shiba's work with Dr. Deming’s concept of 
operational definitions (Doming 1982) led to the outline of a process for 



^ Although the term Decision Support System has generally been applied to problem-solving 
assistance systems using computers (Elam, et al.. 1986), there is evidence that pencil and paper 
delivery systems are just as effective as computerized versions (Cat-Baril & Huber 1987). 
Therefore, I will use the term Decision Support Process (DSP) to refer to a problem-solving 
system without the requirement to include computers. 
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operationally defining customer requirements which the author applied in 
the development of salt-water flyfishing stripping basket.^ 

A review of the Stripping Basket project by a member of the Operating 
Committee of the Center for Quality Management^ (CQM) led to an offer to 
present this material in the CQM's Six-day TQM Course for Senior Executives 
in the summer of 1991. This offer blossomed into a two year collaborative 
effort by representatives of several CQM member companies and MIT to 
apply the Plan-Do-Check- Act cycle (Ishikawa 1985) to the development of 
what eventually became Concept Engineering. 

Mutual Learning 

In the development of Concept Engineering, the company participants 
were equal partners with the researcher in identifying and evaluating the 
investigated problems and solutions. Representatives from four companies 
and MIT were engaged in a continuous relationship over a two year period. 
The members met collectively to discuss objectives and findings and worked 
independently pursuing particular assignments. In stretches, often lasting 
several months, the group met for as much as one full day per week. Interim 
periods were spent implementing and evaluating the results of previous 
decisions. 

During the evaluation periods, it was not unusual for members of one 
company to be present in the product development team meetings of other 
participating companies observing, along side the MIT researcher, the effects 
of proposed solutions. This level of sharing allowed insights into what 



2 The stripping basket, which has been patented and licensed, has been reviewed in the New 
York Times and was widely acclaimed in the flyfishing trade press in 1992 and 1993. 

^ The Center for Quality Management, headquartered in Cambridge MA., is a consortium of over 
thirty organizations d^icated to the pursuit of Total Quality Management. 



16 



worked and didn't work to be rapidly spread among participating companies, 
i.e., an innovation at one company could be applied at another company 
usually in the matter of days or weeks at most. (The resulting rapid feedback 
on process improvement opportunities was a major contributing factor in 
Concept Engineering's development into a complete decision support 
process.) The practice of mutual learning and sharing continues as evidenced 
by two presentations at the February, 1993 Concept Engineering User’s Group 
in which two companies presented innovations and enhancements to the 
Concept Engineering process (Appendix B). 

A significant advantage of practitioner research partners is the ability to 
focus effort on substantive issues. In this research effort, the problems 
investigated were those which product development professionals in the 
firms were facing. "Practitioners often bring the pursuit of irrelevant or ill- 
conceived lines of inquiry to a rapid halt, correcting or refining the questions 
asked in ways that lead to sharper formulation and more productive 
research" (Whyte et al. 1991; p. 54). Additionally, the investment made by 
the organizations in researching existing problems provides a built-in 
incentive for implementing the solutions. This model is in sharp contrast to 
the conventional approach of literature reviews, hypotheses development 
and subsequent search for an organizational setting for testing (Whyte et al. 
1991). 

Elden and Levin (1991) describe this process of participative action 
research as "cogenerative learning". In cogenerative learning, insiders and 
outsiders bring their respective frameworks (understandings) of events 
together to create a shared explanatory framework more powerful than any 
they could have generated independently. Insiders experience the workplace 
directly and have a great deal of specific knowledge of the setting; this 
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knowledge is often tacit and not reflected on. Outsiders (researchers) have 
general knowledge of the field of interest and training in systematic inquiry 
and analysis techniques. This marriage of specific and general knowledge 
provides an opportunity for the creation of new substantive knowledge. 

Concept Engineering Description 

Concept Engineering is a conceptual model, with supporting decision 
aids, for developing product concepts. The process alternates between the 
level of thought (reflection) and level of experience (data) (Kawakita 1991) in 
a way that allows participants to understand what is important to the 
customer, why it is important, how it will be measured and how it will be 
addressed in the product concept. Concept Engineering has five stages each 
with three steps (see figure on the next page). These stages and steps form the 
road map which outlines the conceptual model underlying our product 
concept decision process. 

The model begins with developing a plan for the entire concept 
development process and ends with the selection of the product concept to be 
pursued in subsequent development activities. Within each step, decision 
aids are provided to assist decision making. In some instances, alternative 
decision aids were employed and evaluated for apparent effectiveness in 
providing assistance in the product concept decision process. Effectiveness 
was determined by a consensus opinion of the participants of the Concept 
Engineering research team and/or members of actual product development 
teams. The Plan-Do-Check-Act cycle is continuously applied to the 
development of the conceptual model and supporting methodologies. A 
brief description of each stage, as it currently exists, is provided below. Refer 
to Appendix A for more detailed information. 
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Concept Engineering 
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Stage 1: Understanding Customer's Environment 

The objective of Stage 1 is for the development team to develop 
empathy for the customer, in the actual use environment of the product or 
service. Stage 1 consists of developing a plan for the team's exploration, 
doing the exploration, and using the data collected by the team to develop a 
contextual anchor from the images of the customers' environment. The 
creation of this common mental map of the customer's environment is a 
unique aspect of Concept Engineering. 

The planning process begins with an articulation of the project scope. 
After the scope is outlined, appropriate market segments and customer types 
are identified for investigation. Prior to visiting the selected customers, the 
team members develop an open ended interview guide and interview skills. 
Pairs of team members (usually cross-functional) visit customers and conduct 
the interview at the customer's site and take verbatim notes of customer 
comments and their own observations. Upon completion of the interview 
process, images of the customer’s use environment are selected and analyzed 
with a KJ diagram^ (Ofuji 1990, Kawakita 1991, Shiba et al. 1991a). This "Image 
KJ" is a link to the customer's real world and acts as a contextual anchor in 
the customer's environment for all future product concept decisions. 

Stage 2: Converting Understanding into Requirements 

Stage 2 distills what was learned from the customer exploration into a 
small set of well understood, critical customer requirements. In this stage, the 
Image KJ developed in Stage 1, is used as a contextual anchor in the 
development of requirement statements to ensure they are consistent with the 

^ KJ diagrams structure detailed language (vs. numerical) data into more general conclusions 
using semantic and abstraction guidelines. They are one of a fanaily of tools invented by Jiro 
Kawakita and known as the KJ method (Kawakita 1991). 
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customers' environment. A small set of the vital few from the useful many 
requirements is selected and the relationships between them are analyzed. The 
process and guidelines for linking customer voices to contextual images and 
transforming the voices into requirement statements is unique to Concept 
Engineering. 

The transformation process converts the customer's language, often 
laden with emotion, into a customer requirement statement better suited for 
use in downstream development activities (Ofuji 1990). Each customer voice 
is explicitly linked to an image of the customer's environment and through a 
clearly defined process of successive refinement is developed into customer 
requirement statements/^ The entire team then selects the vital few customer 
requirements from the useful many through a democratic and iterative 
process® of identifying the most important requirements based on their 
respective understandings of the opportunity/The KJ method (Shiba et al. 
1991a) is again employed to develop new insight and team consensus 
regarding the relationships among requirements. 

Stage 3: Operationalizing What You Have Learned 

In Stage 3, the goal is to ensure that the key customer requirements are 
clearly, concisely, and unambiguously communicated in measurable terms. 

The key customer requirements are validated with customers, operationally 
defined in measurable terms and the resulting information is displayed in 
such a way that the relationships between requirements, metrics and 
customer feedback is easily seen. The application of Kano's analysis, described 
in detail in Appendix A, to customer requirements is unique in US concept 
development activities (Kano et al. 1984). 

® The Multi-stage Picking-up Method, another of the KJ method tools, focuses on the most 
powerful statements by eliminating non-candidates in an iterative selection process. 
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/ Two validation methods, self-stated importance assessments and 
Kano's analysis, are employed during this stage to assess customer attitudes 
towards the selected customer requirements. The self-stated importance 
questionnaire is a traditional marketing research technique (Griffin & Hauser 
1992) and indicates the relative importance of requirements. Kano’s analysis 
(Kano et al. 1984) segregates requirements into four categories (Attractive, 
One-dimensional, Must-be, and Indifferent) depending on the relationship 
between changes in functionality and changes in customer satisfaction. 



^y^ditionally, in Stage 3 the team develops and structures® metrics in order to 
measure, quantitatively, requirement realizationy^This stage concludes with 
the development of a Quality Chart and Operational Definitions (Deming 
1986, Hauser & Clausing 1988, Juran 1988, Akao 1990) to integrate customer 
requirement understanding. 



Stage 4: Concept Generation 

This stage marks the transition in the development team's thinking 
from the "requirement or problem space" to the "solution space." In this 
stage the objective is to develop the largest number of potential solution ideas 
possible. Multiple perspectives of the development project are used to 
generate ideas from distinct vantage points. The use of a structured idea 

development process is uncommon in US product concept development. 

/ 

/ The complex design problem is decomposed into smaller, independent 
sub-problems based on the customer's perspective and also from the 
engineering development perspective/'^he team creates, through individual 
and group collaboration efforts, an exhaustive list of ideas (both feasible and 
unfeasible) for each sub-problem; working first from the customer's vantage 

® Tree diagram method relates means to ends, which are in turn means to more general ends, in a 
hierachial relationship (Shiba 1991b). 
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point before exploring the internal engineering perspective. Generated ideas are 
systematically reviewed and enhance^x^^is stage concludes when each team 
member creates their ideal solution concept from the generated list of ideas. 



Stage 5: Concept Selection 

In the final stage of Concept Engineering a product concept is selected 
for downstream development. In this stage, concepts are systematically 
reviewed, compared, combined and enhanced in an iterative process of 
concept development. Concepts are evaluated against customer requirements 
and organizational /environmental constraints. 

In the previous stage, the development team generated a wide array of 
solutions to address collectively the set of customer requirements^/ln this 
stage, the team thinks individually and together, seeks expert help, and 
experiments in the laboratory in an iterative process of combining and 
improving initial solution concepts to develop a small number of superior 
concepty^ The "surviving" complete concepts are evaluated in detail against 
customer requirements and organizational constraints in order to select the 
dominant concept(s^4vhen completed, an audit trail exists for tracing the 
entire decision process from project scope determination through detailed 
concept analysis as the Concept Engineering process is self-documenting. 



Decision Support Processes 

Decision Support Processes are designed to support decision makers in 
the various phases of problem solving. So far, no generalized problem 
solving process exists that has been empirically validated (Mintzberg et al. 
1976, DeSanctis & Gallupe 1987, Sainfort et al 1990). However, Mintzberg and 
colleagues in their classic field study (Mintzberg et al. 1976) of 25 strategic 



23 



(unstructured) decision processes concluded that the decision process has 
three phases: identification, development, and selection. Identification 
consists of both recognition and diagnosis routines. The development phase 
includes two basic routines: search and design. The selection phase consists of 
screening, evaluation-choice and authorization routines. Furthermore, the 
Mintzberg study identifies three sets of supporting routines: decision control, 
communication and political activities, which facilitate the three major 
phases of the decision process. 

In the context of the product concept decision process, Mintzberg et al. 
(1976) empirically developed problem solving process can be redefined to be: 
requirement identification, idea development, and concept selection. I will 
use this framework to illustrate how Concept Engineering is a complete 
Decision Support Process^, the table below outlines the relationships which 
are described in subsequent paragraphs. 



Decision Phase 


C.E. Step 


Decision Aid 


Identification 


1 


Customer Selection Matrix 


- recognition 


2 


Interview Guidelines 


- diagnosis 


3 


Image KJ 




4 


Transformation Process & 




5 


Guidelines 




6 


Multi-stage Picking-up Method 
Requirement KJ 


Development 


7 


Self-stated Importance Assessment 


- search 




Kano’s Analysis 


- design 


10 


Multiple Design Decomposition's 




11 


Idea Generation Process 




12 


Solution Concept Generation 


Selection 


13 


Screening Matrix 


- screen 


14 


Selection Matrix 


- evaluation-choice 

- authorization 




Self-documenting Audit Trail 



7 This argument could also have been made with alternative descriptions of the problem 
solving process, i.e., Sainfort et al. 1990, MacKay et al. 1992. 
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Mintzberg et al. observed that the identification phase consists of both 
recognition and diagnosis activities. They defined diagnosis as "the tapping 
of existing channels and the opening of new ones to clarify and define the 
issues" (p.254). Concept Engineering provides conceptual and methodological 
guidance for clarifying and defining the issues. Stages 1 and 2 deal explicitly 
with exploring the market and converting the knowledge gained in the 
exploration into a well-defined and focused set of customer requirements. 
Specifically, in Stage 1, Understanding the Customer’s Environment, a 
"Customer Selection Matrix" is developed to identify exploration arenas; this 
matrix explicitly includes past, present and prospective customers. Next, 
"Interview Guidelines" are developed to assist the focus of the exploration 
efforts. Stage 2, Converting Understanding into Customer Requirements, 
provides clear guidance in the form of "Translation Guidelines" and 
"Transformation Worksheets" for converting the Voice of the Customer 
information gathered in Stage 1 into unambiguous and nonrestrictive 
Customer Requirements Statements. The vital few requirement statements 
are identified using the Multi-stage Picking-up Method and structured using 
the KJ diagram (Kawakita 1991, Shiba et al. 1991a). 

The Development Phase observed by Mintzberg et al. consists of both 
search and design routines. They indicate four different kinds of search 
behavior: memory, passive, trap and active. In Stage 3, Operationally 
Defining Requirements for Downstream Development, the requirements 
developed and selected in Stage 2 are actively validated with potential 
customers through the use of Self-Stated-Importance questionnaires and 
Kano questiormaires. The Idea Generation step in Stage 4, Concept 
Generation, could conceivably incorporate all four types of search activities. 
The Mintzberg study identified design activities that result in either custom- 
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made or modified solutions. The concluding step of Stage 4 is the generation 
of custom-made solutions that address the set of customer requirements. It is 
possible that constraints imposed on the design team could limit idea 
generation, and thus solution generation, to existing solutions which would 
result in "modification" design activities. 

The Mintzberg study identified three routines in the Selection Phase: 
screen, evaluation-choice, and authorization. The first step in Stage 5, 
Concept Selection, is Solution Screening. In this step a "Screening Matrix" is 
employed to reduce the number of alternatives to a smaller number of 
feasible alternatives. Additionally, each proposed solution is evaluated 
against the customer requirements relative to a pre-selected datum. In the 
second step of Stage 5, Solution Selection, a more analytical comparative 
process is introduced, if necessary, to further assist the development team in 
identifying the dominant concepts. Authorization, the final routine observed 
by Mintzberg et al. in the Selection Phase is not specifically addressed in 
Concept Engineering. However, each step of Concept Engineering is self- 
documenting; some development teams have used their Concept 
Engineering working documents in their project proposal presentations 
before management authorization committees. 

The three routines that support the three central phases of the decision 
process observed by Mintzberg et al. are: decision control, communication, 
and politics. The decision control routine consisted of two basic activities: 
planning and switching. Decision planning consists of "a rough schedule for 
solution, a development strategy, and an estimate of the resources" (p.261). 
The Concept Engineering process is described with a flow chart outlining a 
coordinated set of conceptual steps. Furthermore, in the introduction to the 
Concept Engineering Manual (Burchill et al. 1992) a Gantt Chart displaying 
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the various activities and estimated completion times is provided to assist in 
project planning. Switching "directs the decision maker's attention to the 
next step, to choosing the appropriate routine, such as diagnosis or search,..." 
(p.261). With respect to switching, the Concept Engineering manual also 
provides checklists at the end of each step to assist in determining if a 
minimum set of observable conditions has been met before moving to the 
next set of activities. 

The Decision Communication routines observed by Mintzberg et al. 
include; exploration, investigation and dissemination. Exploration is 
described as a general or passive search for information. The investigative 
routine involves the focused search and research of special-purpose 
information. Dissemination involves communication of information about 
the decision process progress or outcome to ensure eventual acceptance. 
Concept Engineering is geared towards investigative information searches in 
that the objective and recommended information processing approach are 
clearly established for each step of the process. Concept Engineering facilitates 
dissemination by having clearly defined switching points and criteria and the 
self-documenting nature of the tools mentioned previously. 

According to Mintzberg et al. political activities "reflect the influence of 
individuals who seek to satisfy their personal and institutional needs by the 
decisions made in an organization" (p.262). This is consistent with Salancik 
and Pfeffer's (1974) view that power is used in organizations to influence 
decisions concerning the allocation of resources; the more scarce the resource 
the less objective criteria and the more power will be used in obtaining it. 
Additionally, Salancik and Pfeffer state that when there is a disagreement 
about the priorities and consequences of possible actions, decisions can not be 
rationalized. Hickson et al. (1971) propose that "preventive routinization" 
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reduces or removes uncertainty and thus reduces opportunities for the use of 
power. Concept Engineering, which assists all stages and supporting routines 
of the decision process, removes uncertainties, clarifies priorities and the 
relationships between potential actions and objectives. This in turn, increases 
the likelihood for rational decision making thus reducing the opportunity for 
political activities. 

Conclusion 

Concept Engineering is a conceptual model with supporting tools and 
techniques. Furthermore, in relation to the empirical decision structure 
outlined by Mintzberg et al. (1976), Concept Engineering addresses not only 
each phase of the product concept decision process (requirement 
identification, idea development, and concept selection) but also each of the 
supporting routines (decision control, communication, and politics). In these 
respects it should be considered a complete decision support process. 

The development of Concept Engineering, particularly given the active 
participation of corporate product development professionals, provided an 
opportunity to conduct a comparative study of product concept development 
activities. Chapter 2 outlines the research objectives, design and subsequent 
implementation. Chapter 3 presents Inductive System Diagrams as a method 
for developing and articulating grounded, substantive theories for the 
product concept decision process. Chapter 4 presents the evidence, inferences 
and propositions related to management choices in the product concept 
decision process. Chapter 5 outlines management diagnosis opportunities of 
product concept development. Chapter 6 summarizes the findings and 
outline potential future directions. 
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Chapter 2: Research Design 



This research was designed to develop a substantive theory to help 
clarify the product concept decision process and for generating data related to 
the effectiveness of Concept Engineering as a method for developing product 
concepts. A key ingredient for developing both the theory and the method 
was the use of comparison groups. By comparing similar and dissimilar 
groups we could more readily identify key concepts. However, I recognized 
that if some of the comparison groups were stacked in favor of success or 
failure, the conclusions reached could be misleading at worst and delayed at 
best. As a result, I requested randomization controls to address much of this 
bias to provide a stronger foundation upon which to build the method and 
theory. 

In the proposed design, each of three participating companies would 
identify two pairs of development teams. Each pair would be approximately 
similar in scope, demographics, and history. One team from each pair would 
be randomly assigned to use the Concept Engineering process while the other 
team would use Pugh’s Concept Selection process (Pugh 1981) which is 
similar to Stage 5 of the Concept Engineering process. 

The progress and outcome of the research would be assessed in three 
ways. First, field research techniques for observations, interviews and survey 
instruments would be employed to develop an understanding of how and 
why Concept Engineering works. The specific questions pursued were 
expected to change over the course of the research, but based on an 
exploratory cause-and-effect diagram (figure 2.1) they would center around 
the concepts of: structured design process, dear customer requirements, and 
organizational commitment. 
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Second, objective process measures developed by the CQM Research 
Committee (CQM 1992) would be used at both the requirement generation 
and concept selection stages. Finally, subjective assessments by relevant 
company officers would be made of the performance of each team. 

Validity Threats 

Every research design should attempt to preclude as many validity 
threats as possible. Internal validity represents the degree of confidence that 
the input changes actually caused the observed outcomes. External validity 
reflects the degree of confidence that the results can be generalized to groups 
other than those studied. Construct validity addresses how well the research 
measures what was intended to be measured (Cook & Campbell 1979, Kidder 
& Judd 1986). The design outlined above, and agreed upon by representatives 
(a chief operating officer, a general manager and a director of product 
development) of the participating companies, attempted to minimize each of 
these validity threats. 

External Validity 

The nature of the companies participating in the study necessarily 
imposes some threats to external validity, or the ability to generalize the 
findings. Specifically, all of the Concept Engineering teams were fairly small, 
core teams of six to eight members; although the full development team 
could be substantially larger. Additionally, all participating companies 
considered themselves to be "high-tech" and were members of the Center for 
Quality Management (CQM). Membership in the CQM requires a strong CEO 
commitment to Total Quality Management (TQM) and there is considerable 
training and emphasis on the use of many of the techniques (e.g. KJ diagrams. 
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Tree Diagrams) employed in Concept Engineering. Additionally, TQM 
emphasizes standardization and process orientations which may have 
affected the acceptance of the process by development team members. Finally, 
the design techniques introduced in this study represent only a small subset 
of the potential development process enhancements. These factors limit the 
ability to generalize the conclusions of the study. 

Construct Validity 

The principle of triangulation, well known in navigation, applies to 
research as well. At sea, any three measures of location taken by different 
methods, i.e. satellite versus radar, will position you at three different points 
on the map. Your true location is more likely to be within the triangle 
formed by the three points than exactly at any one point. Articles and text 
books on research methodology emphasize the importance of having 
multiple ways of measuring the constructs of interest (Cook & Campbell 1979, 
Kidder & Judd 1986, Blackburn 1987, Jick 1979). In the research design of this 
study, multiple assessment methods, some qualitative and some quantitative, 
were identified for use. 

Internal Validity 

This design was structured to address many potential threats to 
internal validity in order to increase the degree of confidence that the process 
changes actually caused observed changes in the outcomes, if any. 
Compensating Treatment 

Some threats to internal validity, which could be present in any study 
that supplies a favorable treatment to one group and not to the other, revolve 
around the reaction of the "no-treatment" group. On the one hand, they 
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could try harder than usual through a heightened sense of rivalry, and on 
the other they could become demoralized and not work as hard. An accepted 
device for addressing these threats is to provide the "no-treatment" group 
with some form of treatment (Cook & Campbell 1979, Kidder & Judd 1986, 
Blackburn 1987). In this study we intended to provide the non-Concept 
Engineering groups with Pugh's Concept Selection process (Pugh 1981). 
Pugh's process would be recognized as a development process enhancement 
in each of the participating companies. 

Experimenter Expectancies 

An additional threat to the validity of the study comes from the 
expectations of the person delivering the intervention. It has been shown 
that the administrator of the intervention can unwittingly bias the results 
provided by subjects (Cook & Campbell 1979, Kidder & Judd 1986). In this 
study, a graduate student was hired and trained as a research assistant to 
collect data on some of the teams. The research assistant was not provided 
with training in Concept Engineering and thus could provide a control 
against some forms of bias which may have been introduced by the author. 
Randomization 

The importance of randomizing treatment assignment was a critical 
component of the design's defense against the various threats to internal and 
external validity inherent in this study. Specifically, given the availability of 
concurrent, co-located control groups, many threats to validity, such as 
history (events that occur during the experiment unrelated to the treatment), 
testing (the impact of the measurement or observation process on the 
subjects), and instrumentation (the process of collecting data), could be 
compensated for through the design. However, the presence of a control 
group does not address the threat to validity from selection, the process used 
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to assign groups to a treatment condition. Without random assignment of 
conditions, the selection threat opens the door to a multitude of plausible 
rival hypotheses which could account for any observed differences. (Cook & 
Campbell 1979, Kidder & Judd 1986, Blackburn 1987) 

Research Implementation 

The actual implementation fell far short of the research design. While 
this clearly has implications for validation efforts originally designed for the 
study, it did not seriously disrupt the research objective of developing a 
grounded, substantive theory for the product concept decision process. 
However, the trials and tribulations experienced in this study are valuable in 
highlighting some of the difficulties associated with experimental research 
designs in organizational settings. 

All three companies that agreed to participate in the study in the fall of 
1991 sent representatives to the two-week training session in January 1992. 
One company (hereafter referred to as Company 1) began their first Concept 
Engineering effort in February 1992, the second company (Company 2) began 
their first Concept Engineering team in April 1992, and the third (Company 3) 
began in May 1992. It was immediately obvious that the first teams were not 
assigned on a random basis. In Companies 1 and 2 the appropriate managers 
selected an initial team they felt had a high likelihood of success. In 
Company 3, although it was not immediately apparent, the selection and 
staffing of the first team created a high likelihood of failure. This conclusion 
is validated by comments from senior managers in Companies 1 and 2 who 
specifically stated that they needed an initial success and in comments from a 
vice president of engineering in Company 3, who "felt" Concept Engineering 
was a ploy by Marketing to shift their work to Engineering. In short, random 
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assignment to address some of the traditional threats to validity (selection, 
maturation, etc.) did not take place in this study. 

The original design assumed that many teams would be working 
simultaneously; the calendar time associated with each team was expected to 
be approximately four months. In execution, each company started the first 
Concept Engineering effort and then waited for preliminary results before 
committing itself to support a second team. Furthermore, each team took 
about six calendar months to complete its work. This delay had enormous 
implications for the scope of the research given the available time of the 
primary researcher. In hindsight, it became clear that companies would be 
hesitant to commit a second team until the first team could be evaluated, at 
least provisionally. The length of time required for each team to complete its 
work was a surprise. The largest contributing factor to the increase in project 
time was delay time before starting. Once the decision was made for a team to 
apply Concept Engineering, several months might pass before meaningful 
effort was applied to the project. This was primarily due to other project 
commitments of team participants. This problem resulted in fewer Concept 
Engineering teams to be available for the study than had been intended in the 
design. 

With respect to the control groups, the first and second companies also 
provided a non-Concept Engineering comparison team in the spring of 1992. 
These teams were assigned on the basis of availability rather than on 
matching characteristics of scope, demographics, etc. In company 1, the 
comparison investigation was short-lived; the project literally exploded in the 
laboratory after two months. Subsequently, in Company 1, the division 
director was impressed enough with the results of the first Concept 
Engineering development team that he declared that all subsequent 
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sponsored development efforts would use Concept Engineering. In company 
2 , the investigation of the comparison team (not well matched in scope) did 
proceed through to design approval, although the team did not use Pugh's 
concept selection process. In company 3, the chief operating officer carried out 
an extensive study (an entire week, including the weekend, of his time) of 
Concept Engineering and declared that all company-sponsored development 
efforts would be required to use it in order to proceed through the company's 
Product Review Board process. As a result of these differences in approach in 
the three companies, the study of matched comparison groups called for in 
the research design did not materialize. 

Ultimately, the number and nature of cases investigated was 
significantly fewer than anticipated. Therefore, any attempts to evaluate the 
relative effectiveness of Concept Engineering are now subject to considerable 
threats from rival plausible hypotheses. However, I was able to observe 
extensively five development teams in four companies that used Concept 
Engineering and two development teams in two companies that did not. In 
addition, in Companies 1 and 3, it was possible to make historical 
comparisons with the previous project completed by development teams 
assigned to use Concept Engineering. For each development team studied, I 
typically attended every scheduled meeting, approximately 80 hours per team, 
and conducted two to three in-depth open-ended interviews with each 
member of the team and their managers; each interview lasting at least one 
hour. Therefore, although they lacked random assignment, the available 
teams did provide a rich comparative setting, with many similarities and 
dissimilarities, to explore for theory generation. 
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The Theory Generation Study 

In theory generation research, data collection and analysis are 
conducted concurrently (Glaser & Strauss 1967, Barton & Lazarsfeld 1969, 
Miles & Huberman 1984, Schein 1987). “Qualitative research in general and 
theory generation in particular, is essentially an investigative process, not 
unlike detective work. Observing one class of events calls for a comparison 
with a different class. Understanding one relationship reveals several facets 
which have to be teased out and studied individually. The theory is 
developed in large part by contrasting, comparing, replicating, cataloguing, 
and classifying the subject of the study" (Miles & Huberman 1984; p.37). 
Without joint data collection, coding, and analysis, the subtleties in the area 
of study, and opportunities to investigate them, can be lost. As a result, the 
evolving nature of desired information precludes the establishment of 
detailed, pre-specified sampling plans (Glaser & Strauss 1967, Barton & 
Lazarsfeld 1969). In the words of C.I. Lewis (1929) "Knowledge begins and 
ends in experience; but it does not end in the experience in which it began." 

Glaser and Strauss (1967) make an additional distinction between 
sampling required for theory development and theory verification. 
Theoretical sampling, sampling designed to develop rich comparative 
settings, is conducted to identify and investigate variables and their 
interrelationships in the development of theory. Statistical sampling is 
conducted to collect evidence to be used in descriptive or verification studies. 
As a result, they state that the researcher generating theory need not use 
random sampling techniques. 
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Participant Observation 

Forrester (1992; p.57) states the professional literature emphasizes how 
decisions should be made rather than how they are made and "there is not yet 
an adequate literature on what constitutes the practice of identifying decision- 
making policy." Forrester's observation on the general decision making 
process is consistent with specific research findings on the product 
development decision process which consistently identify large differences 
between product development theory and practice (Cooper & Klienschmidt 
1986, Gupta & Wilemon 1990, Mahajan & Wind 1992). In Forrester's view 
(1992; p. 52), "perceptive observation, searching discussions with persons 
making the decisions, study of already existing data, and examination of 
specific examples of decisions and actions all illuminate factors that influence 
decisions." 

"Technically a 'qualitative observation' identifies the presence or 
absence of something, in contrast to 'quantitative observation,' which 
involves the degree to which some feature is present" (Kirk & Miller 1990; 
p.9). Participant observers gather data in the daily life of the organization 
studied (Becker 1969). Two approaches to participant observation were 
combined in this research, grounded theory and action science. In grounded 
theory, the goal is to develop theory, intimately tied to the data, which 
explains, interprets and predicts what is happening in an area of investigation 
(Glaser & Strauss 1967; p.). The action scientist has as a goal, the development 
of theoretical constructs simple enough to be usable while enabling the actor 
to grasp all the relevant features of the situation (Argyris et al. 1985; p.78). 

This strategy has provided the opportunity to generate a substantive theory, 
intimately tied to actual practice, which provides insight and access to 
practitioners in the development of product/ service concepts. 
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Action Science 

In action science, the emphasis is on obtaining or improving basic 
knowledge while solving practical problems J Action science subscribes to 
the view that one of the best ways to understand the world is to try to change 
it. This perspective leads to a direct challenge of the normal science premise 
that the primary objective of science is to describe reality. However, action 
science does subscribe to several features of normal science including 
intersubjectively verifiable data, explicit inferences, disconfirmable 
propositions and public testing (Argyris et al. 1985). 

One of the primary goals of the action science perspective is to 
distinguish between espoused theories and theories-in-use. Espoused 
theories are those that an individual claims to follow. Theories-in-use are 
those that can be inferred from action. The objective is to make explicit the 
largely tacit propositional logic of the form 'Tn situation s (as the actor 
constructs it), do a to achieve consequence c." This means that the research 
must elicit data on what individuals actually say and do as they interact, as 
well as data on what they are thinking and feeling at the time of their actions 
(Argyris, et al. 1985; p.). The ability of the researcher to recognize the 
possibility for inconsistencies between theories-in-use and espoused theories 
depends in many respects on the familiarity of the researcher with the daily 
life of the organization; hence the importance of the participant observation 
research approach. 

The action science requirements for disconfirmable propositions and 
public testing ask that propositions or hypotheses be characterized by features 
that allow practitioners to disconfirm them. These include making 

^ By analogy, this is similar to the early Operations Research work during World War II 
(Argyris, et al.. 1985). 
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propositions public, providing the directly observable data on which they are 
based, making them connectable to these data, and designing conditions 
conducive to testing them for validity (Argyris et al. 1985; p.233). Edgar 
Schein in his monograph, "The Clinical Perspective in Fieldwork" (Schein 
1987; p.29), states that the clinician typically starts with an "action research" 
model i.e. the assumption that one cannot understand a human system 
without trying to change it. As a result, the clinician learns about some of the 
most fundamental dynamics that operate in organizations, and the power of 
such work is its ability to provide better variables and better understanding of 
system dynamics than other research methods. This leads to the conclusion 
that perhaps the best use of clinical work may be in the construction of 
variables and theoretical models (Schein 1987, Blanck & Turner 1987). 

Grounded Theory 

Grounded theory approaches to generating hypotheses are 
characterized by the use of an exhaustive (and exhausting) data-coding and 
memo-writing regimen, as well as the use of the constant comparison 
method of analysis. A grounded theory development process generally 
consists of the following activities: 

1) The researcher starts by coding each incident in his data for as many 
categories of analysis as possible. While coding an incident, the researcher 
attempts to compare this incident with all other incidents in the same 
category. 

2) The researcher regularly stops to record in "theoretical memos" his or her 
thoughts on the developing theory. 
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3) As the coding continues, the unit of comparative analysis changes from 
comparison of incident with incident to comparison of incident with the 
accumulated knowledge of the category. 

4) The accumulated knowledge is integrated into a unified whole. 

5) The theory is solidified as major modifications become fewer, non-essential 
categories are pruned, and higher level concepts are abstracted from the 
detailed categories previously developed from the data (Glaser 1965, Glaser & 
Strauss 1967, Glaser 1978, Strauss 1987). 

Constant Comparison Method 

In the constant comparison method, the objective of the sampling 
process is to allow for comparisons of differences and similarities among the 
units of analysis. This process of analyzing the similarities and differences 
produces the dense category development essential to well grounded theory 
generation. Minimizing differences among comparison groups increases the 
likelihood that a lot of information is available for developing of the basic 
properties and conditions of a category. Identifying similar data under 
comparison conditions of maximum differences identifies the fundamental 
explanatory variables. To integrate these variables into theory requires 
investigating the causes, consequences and constraints of these variables also 
under comparison conditions of maximized differences (Glaser & Strauss 
1967; p56-58). 

Variable Development 

One of the strengths of grounded theory methods is the coding process 
for category development (Glaser & Strauss 1967, Glaser 1978, Strauss 1987). 
"The code conceptualizes the underlying pattern of a set of empirical 
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indicators within the data. Coding gets the analyst off the empirical level by 
fracturing the data, then conceptually grouping it into codes that then become 
the theory which explains what is happening in the data" (Glaser 1978; p.55). 
The process begins with "open-coding", a line by line analysis of the data 
which is diametrically opposite to the process of coding with preconceived 
codes. In open-coding the analyst attempts to code the data in as many 
different ways as possible. The analyst constantly looks for the "main theme", 
for what appears to be the main concern of or problem for the people in the 
setting (Strauss 1987; p.35). As the analyst’s awareness of the central 
problem(s) emerges, they alternate open coding with very directed "axial 
coding". Axial coding consists of analysis done around one category at a time. 
As core variables begin to emerge, the analyst employs "selective coding" to 
focus coding to only those variables that relate to core variables in sufficiently 
significant ways to be used in parsimonious theory. In all 10 to 15 codes are 
typically enough for a monograph on a parsimonious substantive theory 
(Strauss 1987; p.32). 

Theory Generation Research Validity 

In verification research to test a hypothesis, the investigator must 
already know what it is they are going to discover (Kirk & Miller 1990). In 
theory generation research, by definition, the researcher does not know what 
they are going to discover. The relatively small sample sizes and lack of 
reliance on random sampling techniques associated with the theoretical 
sampling requirements of grounded theory methods generate conflict with 
many of the traditional tests of validity outlined by Cook and Campbell (1979). 
As a result, a fundamental issue of theory generation research is how to 
express the validity of the developed theories. 
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Glaser and Strauss (1967) discuss the four properties any grounded 
theory must have for practical application. The theory must fit the 
substantive area in which it will be used — the concepts and hypotheses 
supplied by the theory are closely tied to the data. Second, it must be readily 
understood by people in the area — it will make sense to the people working 
in the area. Third, it must be sufficiently general to be applicable in diverse 
situations — the level of abstraction must be sufficient to make a variety of 
situations understandable but not so abstract as to be meaningless. Finally, 
the theory must allow the user partial control over structure and process — 
the theory must contain sufficient concepts and their plausible interrelations 
to allow a person to produce and predict change. In short, the theory can be, 
and is, used by practitioners to guide what they do. 

Argyris, et al.. (1985) also propose four criteria for testing the validity of 
a theory. First is intersubjectively verifiable data — competent members of 
the scientific community should be able to agree at the level of observation, 
even if they disagree at the level of theory. The second criterion is explicit 
inferences — the logic that connects theory and observation should be 
explicit. Third is the use of disconfirmable propositions — the results of 
observations must relate to the acceptance or rejection of the theory. Finally 
is the concept of public testing — the users of a theory test its validity by 
comparing actual and predicted consequences following a change in their 
actions based on the research. 

From the Clinician's perspective Schein (1987) states that the validity of a 
theory can be determined by its ability to predict the response to an intervention. 
The ethnographic view of validity emphasizes the issues of replication and 
internal consistency (Van Maanen 1983). 
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Walter Shewhart, the acknowledged developer of statistical process 
control, may have said it best when he wrote: "there is an important distinction 
between valid prediction in the sense of a prediction being true and valid 
knowledge in the sense of a prediction being justifiable upon the basis of 
available evidence and accepted rules of inference" (Shewhart 1938; p.)- 
Shewhart (1938) points out that it is possible for predictions to be valid even 
when the knowledge supporting them is not. Similarly, valid inferences can be 
made from faulty evidence. Therefore, if theories result in testable predictions, 
then the validity of theory generation research can be judged on the basis of its 
evidence, inferences and predictions. 

Revisiting the validity criterion outlined above it would appear that 
Schein is concerned primarily with prediction. Van Maanen's concerns seem 
related to evidence and inferences. Glaser and Strauss appear to address 
evidence and inference but not prediction; in addition they are concerned with 
generalizability and user accessibility. Argyris, et al. appear to address evidence, 
inference and prediction. These observations are summarized in the table below. 



Glaser & Strauss 


Argyris, et al.. 


Van Maanen 
Schein 


Shewhart 


Fit 


verifiable data 


replication 


evidence 


Understanding 


explicit inferences 


internal 

consistency 


inferences 




disconfirmable 

propositions 


prediction 


prediction 




public testing 






allows for control 


Jfr Jfr ♦ ♦ 






general 

applicability 









These three concepts: evidence, inferences and predictions, constitute a 
set of requirements which, if addressed in theory generation research, would 
allow researchers to observe and distinguish both the validity of the 
hypotheses (predictions) and the validity of the theory creation process 
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(evidence and inference). An important caveat is drawn from Kuhn's (1962) 
arguments on how paradigms affect our abilities to interpret the arguments of 
others, i.e. because we interpret issues from our paradigm not others, it will 
be difficult for distinct schools of thought to agree on whether any given piece 
of "knowledge" is valid because the accepted rules of inference are different. 
However, at a minimum, it should be possible to assess whether the 
hypothesis or prediction itself is valid. 

Finally, although not essential from a validity perspective, I would also 
encourage researchers to strive for user accessibility as a criterion for 
substantive theory construction. For practitioners to use a theory they must 
understand how variables under their control relate to the system of interest; 
if a theory describes a system without providing practitioners access it won't 
be used. 

Conclusion 

The original design had accommodations to many recognized threats 
to validity, but in the execution of the research the researcher was unable to 
ensure that the research was implemented as designed. A researcher simply 
does not have sufficient influence to control the operational requirements of 
organizations. On the other hand, the research project still provided a 
valuable opportunity to conduct detailed investigations of the development 
process in a comparative setting. This environment supports the goal of 
generating a substantive, grounded theory to provide leverage to 
practitioners in clarifying the product concept decision process. However, the 
methodologies appropriate to exploring this setting are not widely accepted as 
mainstream operations management (or social science) research methods. 
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Chapter 3: Inductive System Diagrams 



Social scientists, over the past sixty years, have developed 
methodologies to generate theory through an inductive process based on the 
intensive analysis of a small number of data sources. However, a difficulty 
associated with these, and most styles of qualitative theory development, is 
conveying credibility. The researcher usually presents only enough data to 
project credibility which is often not enough to allow for verification by 
others. A theory which is not clearly stated, and not well integrated, has a 
reduced likelihood of being accepted (Glaser 1965). 

Inductive System Diagrams combine aspects of Grounded Theory 
methods and System Dynamics. Grounded theory approaches are used to 
develop the variables which have a great deal of explanatory power and are 
intimately tied to the data. The cause and effect relationships among these 
variables are then shown using causal-loop diagramming techniques from 
System Dynamics. This combination of grounded theory and causal-loop 
diagramming allows researchers to generate and communicate substantive 
theories intimately tied to the data. 

Grounded Theory 

Grounded theory approaches to generating hypotheses are 
characterized by the use of an exhaustive (and exhausting) data-coding and 
memo-writing regimen, as well as the use of the constant comparison 
method of analysis. In the constant comparison method, the objective of the 
sampling process is to allow for comparisons of differences and similarities 
among the units of analysis. This process of analyzing the similarities and 
differences produces the dense variable development essential to well 
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grounded theory generation. Minimizing differences among comparison 
groups increases the likelihood that a lot of information is available for 
developing of the basic properties and conditions of a variable. Identifying 
similar data under comparison conditions of maximum differences identifies 
the fundamental explanatory variables. To integrate these variables into 
theory requires investigating the cause, consequences and constraints of these 
variables also under comparison conditions of maximized differences (Glaser 
& Strauss 1967; p56-58). 

Variable Development 

One of the strengths of grounded theory methods is the coding process 
for category development (Glaser & Strauss 1967, Glaser 1978, Strauss 1987). 
"The code conceptualizes the underlying pattern of a set of empirical 
indicators within the data. Coding gets the analyst off the empirical level by 
fracturing the data, then conceptually grouping it into codes that then become 
the theory which explains what is happening in the data" (Glaser 1978; p.55). 
The process begins with "open-coding", a line by line analysis of the data 
where the analyst attempts to code the data in as many different ways as 
possible. As the analyst's awareness of the central problem(s) emerges, open 
coding alternates with very directed "axial coding". Axial coding consists of 
analysis done around one category at a time. As core variables begin to 
emerge, the analyst employs "selective coding" to focus coding to only those 
variables that relate to core variables in sufficiently significant ways to be used 
in parsimonious theory. In all 10 to 15 codes are typically enough for a 
parsimonious substantive theory (Strauss 1987; p.32). 
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Open Coding 

By definition in theory generation research, the essential variables are 
not known; open coding is the start of the variable development process. 
During open coding each sentence is explored for as many possible concepts as 
possible. When coding the concept, it is assigned a variable name which is 
closely linked to the supporting data. Questions related to the occurrence of 
the concept are generated. These generative questions build sensitivity for 
future use in making comparisons when the next occurrence of the concept is 
encountered (Glasser 1978, Strauss 1987). An example, from my field notes is 
provided below: 



"This was a decision node in the conception of the 
product which was not made by systematic analysis.” - 
R&D manager 

Decision node . What is a decision node? How many are 
there? What are the necessary conditions for an event to 
be considered a decision node? Who initiates the 
decision? Who ratifies the decision? Who monitors 
them? 

Conception . When is a product conceived? What is the 
gestation period like? I can think of lots of analogies 
here, prenatal care, miscarriages, etc. ... 

Systematic Analysis . What constitutes systematic? 
unsystematic? When does one favor one over the other? 
Assuming systematic is preferred; how does one get 
away with unsystematic analysis? 



The open coding process generates a large number of variables quickly. 
Therefore it is necessary to reduce codes in use. The reduction occurs through 
a process of abstraction (Hayakawa 1990). In abstraction, variables which 
convey similar concepts are grouped together and a variable name, which 
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captures the essence of the common concept, is selected to be used in all 
references to this concept. In some cases, one code in the grouping represents 
the best label for the concept and it can be used for the variable name. In 
other cases, it is necessary to create a variable name which captures the 
common concept. In the example below, systematic analysis was selected as 
the variable name which best captured the common concept in all six codes. 

systematic analysis 

- design constraint tradeoff 

- performance comparisons 

- conscious dimension sacrifice 

- tradeoff equation 

- doing homework 

By investigating events under similar conditions, those concepts which 
are common in different settings represent the initial pool of potential 
explanatory variables. (It is highly probable that the final set of variables could 
be substantially different than the initial set.) Axial coding is used to develop 
better insight into these variables. 

Axial coding 

Axial coding represents an attempt to identify the causes, consequences and 
constraints of a variable under investigation. It is designed to build substantial 
knowledge about the selected variable and other variables it relates to (Glasser 1978, 
Strauss 1987). In studies where both participant observations and interviews are 
conducted, it can be very productive to conduct a "Causes, Consequences and 
Constraints" structured interview with participants as soon as possible after 
observation of the concept of interest. Reviewing existing field notes for evidence of 
causes, consequences and constraints can also be productive as the following 
example shows: 
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"Going back and doing the correlation effort yielded the same 
numbers and is documented. This gives us triple verification 
of what we are doing. So Tm willing to sign." - design engineer 

Systematic Analysis causes Traceability causes Confidence 



Selective Coding 

When a variable begins to stand out as being the core category, as having 
extra-ordinary explanatory power, it is selected for focused coding. Coding 
activities are focused exclusively on the selected variable and the other 
variables with which it has significant relationships. All available data should 
be considered for review in selective coding (Glasser 1978, Strauss 1987). In this 
study, the KJ method (Kawakita 1991, Shiba et al. 1991a) was used to investigate 
core variable relationships, ( see Appendix D for examples.) 



Iteration 

Cycling back and forth between open, axial and selective coding occurs 
regularly early in the investigation and gradually decreases as the research 
progresses (Glaser & Strauss 1967, Strauss 1987). For example, at any time during 
this process insight regarding the variables or related inferences may occur. 
When this happens, immediately stop and write a "theoretical memo" before 
continuing or at a minimum make an appropriate annotation in the field notes 
as shown below (Glaser 1965, Glaser & Strauss 1967, Glaser 1978, Strauss 1987). 

"It is becoming increasingly important because this 
process is taking a long time, not just a long elapsed 
time because it is not calendar time, but in terms of 
people time it is extensive" - marketing manager 

Systematic Analysis causes Lab or R equirements- 
Memo: Labor Ayailability constrains Systematic Analysis 
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The insight (captured in writing first) can trigger a change in coding strategy. 
In the example above, the data show evidence that Systematic Analysis causes 
Labor Requirements . A logical inference, not supported by the evidence, is 
that Labor Availability could constrain Systematic Analysis . Accordingly, 
additional theoretical sampling and/or more open coding connected to the 
concept of labor could follow from this insight. In another example, an 
integrating diagram can be developed on the basis of axial coding. Analysis of 
preliminary diagrams can (often) identify inferences regarding variable 
relationships which are not supported by available evidence. This can trigger 
additional theoretical sampling, open coding and/or axial coding as required 
to explore the proposed relationship. 

System Dynamics 

Forrester (1968; p.1-2) argues that a "structure (or theory) is essential if 
we are to effectively interrelate and interpret our observations in any field of 
knowledge." A hierarchical framework for identifying the structure of a 
system has been identified and developed in the system dynamics field (see 
for example: Forrester 1968, Goodman 1974, Randers 1980, Richardson and 
Pugh 1981). These principles of system dynamics can be applied to decision 
processes to develop their underlying structure (Forrester 1968, Goodman 
1974, Randers 1980, Sterman 1989). 

At their highest level, systems can be described as being open-loop or 
closed-loop (Forrester 1968). Forrester identifies open systems as being 
characterized by current performance is not influenced by past behavior; 
open-loop systems do not observe, and therefore react, to their own actions. 
Closed-loop systems, on the other hand, are characterized by the feedback 
from past performance influencing current actions. Decision processes are 
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closed-loop systems as they are imbedded in a feedback loop; the decision, 
based on the available information of the state or condition of the system, 
controls an action influencing the system condition, which generates new 
information, which is used to modify the next decision (Forrester 1968; p.4-4). 

Interconnecting feedback loops are the basic structural elements in 
systems which generate dynamic behavior (Forrester 1968, Goodman 1974). 
"Feedback loops are a closed path connecting in sequence a decision that 
controls action, the level of the system, and information about the level (or 
condition) of the system, the latter returning to the decision-making point" 
(Forrester 1968; p.1-7). However, at a lower level of hierarchy, feedback loops 
contain a substructure composed of two types of variables — levels and rates 
(Forrester 1968). The level (or state) variables describe the condition of the 
system at any particular time while the rate variables tell how fast the levels 
are changing (Forrester 1968). 

To illustrate these points, consider the decision process of filling a glass 
from a beer tap. When we are thirsty and the glass is empty, the decision is to 
open the tap fully. As the level of beer in the glass approaches the top, we 
decide to gradually close down the tap, reducing the rate at which beer enters 
the glass so that the tap is closed when the glass is full and no beer is wasted. 

Causal-loop Diagrams 

Causal-loop diagrams identify the principal feedback loops in a system 
without distinguishing between the nature, i.e. level or rate, of the 
interconnecting variables (Goodman 1974). Goodman (1974) outlines the 
steps of developing a causal-loop diagram as follows: 

1. establish the pairwise relationships of relevant variables; 

2. ascertain the polarity of the causal pairs; 



53 



3. fit together the causal pairs into closed loops; and 

4. test for loop polarity. 

Through this process, the causal-loop diagram allows the analyst to integrate 
the variables they have developed, explicitly state the inferences they are 
making and clearly communicate their hypotheses regarding the dynamics 
associated with the structural relationships of the system. 

Pairwise variable relationships are diagrammed with directed arcs. 

Arcs are used to connect the factors which influence each other; the arrow 
indicating the direction of influence. Each arc is annotated with an indication 
of the causal change (polarity) between the two factors.^ An "S" indicates that 
the two factors move in the same direction, i.e., all other things being equal, 
as one variable increases the other variable also increases. An "O" indicates 
that variables move in opposite directions, i.e., all other things being equal, as 
one factor increases the other factor decreases. These pairwise arcs can then be 
connected to form feedback loops. 

There are two basic types of feedback loops, reinforcing (positive) and 
balancing (negative) feedback loops which are used to explain the dynamics of 
complex situations (Forrester 1968, Goodman 1974, Randers 1980). 

Reinforcing loops promote movement, either growth or decay, by 
compounding the change in one direction. Balancing loops resist change in 
one direction and try to bring a system back to a specified goal or equilibrium 
state. These two simple structures can be combined in an large variety of 
ways into casual loop diagrams which can be used describe complex systems. 



^Goodman (1974) uses V and to indicate positive and negative polarity. Senge (1990) and Kina 
(1992) advocate the use of 'S' and 'O'. 
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ISD Step by Step Methodology 

The development of Inductive System Diagrams starts with identifying 
the central variables and concludes with mapping their relationship through 
causal loop diagrams. An modification of a step-by-step process for 
developing system diagrams developed by Burchill and Kim (Burchill & Kim 
1993) is outlined below; 

Step 1: Selecting a Variable 

The focus of the investigation is established by identifying significant 
(core) variables (categories) and their symptoms. The initial selection of a 
variable is decided by its apparent explanatory ability or central importance in 
the events being studied. (This implies that considerable open coding and 
comparative analysis has been conducted by the researcher.) This can be done 
through axial coding - the process of specifying the varieties of conditions 
and consequences associated with the appearance of phenomenon referenced 
by the variable (Strauss 1987;64). 

Step 2; Identifying Causes and Consequences 

After a significant variable is identified, the next step is to identify 
other variables closely related to it. The data are analyzed to identify key 
factors which appear to drive or be driven by the selected variable. This can be 
accomplished by selective coding, wherein all other subordinate variables and 
their dimensions become systematically linked to the selected variable. 
(Strauss 1987) 

Step 3: Describe Factor Relationships 

After key factors associated with a variable have been identified, their 
interactions are diagrammed as causal-loop diagrams. The pairwise directed 
arcs developed during axial and selective coding are integrated into a closed 
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system. There are usually many variables to explore and it doesn't matter 
which one is selected first assuming all will be investigated. 

Step 4: Check Diagram Consistency 

The diagrams should be compared to the collected data to ensure they 
are grounded in the available facts. Often early diagrams contain links which 
are not supported by the presented evidence. If upon review, the researcher is 
confident the loop reflects the system dynamics, additional theoretical 
sampling or coding is necessary to ensure the theory remains "grounded" in 
the available data. Additionally, the diagrams should be investigated for 
"leaps of logic", i.e.. can the diagram describe the patterns of events without 
explanation. Finally, the diagram is reviewed to ensure factor labels are at the 
same level of abstraction (Hayakawa 1990). For example, "Design Constraint 
Tradeoff and "Performance Comparison" would be at the same level of 
abstraction while the abstracted category, "Systematic Analysis" would be at a 
higher level of abstraction. 

Step 5: Integrating Causal-loop Diagrams into an Inductive System Diagram 

After all significant variables have been diagrammed, the individual 
causal-loop diagrams are combined to articulate the underlying structure or 
theory. A central theme is developed using a clearly dominant (core) variable 
or by linking variables which are common to multiple causal-loop diagrams. 
Remaining causal-loop diagrams are incorporated into the central theme. 
Variables may be combined and re-labeled at a higher level of abstraction 
(Hayakawa 1990). Additionally, low impact loops are eliminated to simplify 
the diagram. This integrated ISD is validated for logic flow, abstraction levels 
and consistency with the data. 
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Product Development Study Example: 

An example of the use of ISD in the development of a substantive 
theory for product development activities follows. The specific coding and 
analysis examples come from teams using the Concept Engineering method. 
All field notes were exhaustively coded and analyzed (an average of three 
hours of off-site effort for every hour of recorded notes) by the author and/or 
a research assistant. Additionally, much of the coding and analysis was 
reviewed by colleagues in a Field Research Methods Seminar. 

One team went from kick-off to product requirement determination in 
less than two months and on to final product concept selection in only two 
more months - considerably faster than historical performance. As a result. 
Development Time was selected for focused investigation (theoretical 
sampling/ axial coding). Examples of relevant quotes from field notes (italics) 
are provided to illustrate the ISD process. 

"(On the previous project) This process would have provided a clearer 
vision^, a straighter path to the end result^. I see the process saving tim^ by 
eliminating missteps^." - Engineering Development Manager 



Coding this statement for variable development might create categories 
for: 1) Design Vision Clarity, 2) Straighter Path, 3) Development Time, and 4) 
Missteps. Straighter Path and Missteps are conceptually similar and at a 
higher level of abstraction could both be dimensions of the category 
Misdirected Effort. These variables can be diagrammed as follows: 



Design 

Vision 

Clarity 




Misdirected ^ Development 

Effort 5 Time 



This diagram indicates that as Design Vision Clarity increases Misdirected 
Effort decreases causing Development Time to also decrease. 
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The constant comparison method employed in a Grounded Theory 
approach requires that events be compared to other incidents in the same 
category. Accordingly, the following incident, from the same team, which relates 
to Development Time was compared to the instance above. 

"Someone that has buy-in^ understands the how and why and can explain to 
other people horizontally or vertically^ . Along with buy-in is a belief or passion^. 
I think that where there is passion there is ownership and those two combined'^; 
when they exist in the same group of people and the team encounters problems 
they don’t last^. The team fixes it and moves on^." - Marketing Product Manager 



Coding this statement for variable development might create categories for: 
1) Buy-In, 2) Design Objective Understanding, 3) Passion, 4) Ownership, 5) Design 
Problem Resolution and 6) Development Progress. To simplify coding, Buy-In, 
Passion, and Ownership can be combined into an abstracted category Conviction. 
Additionally, Design Objective Understanding is conceptually similar to the 
variable Design Vision Clarity in the diagram above and is abstracted into the 
variable Design Objective Clarity. Development Progress is conceptually similar 
to the variable Development Time; Development Time will continue to be used 
as it is less ambiguous then Development Progress. The resulting diagram, 
integrated with the previous diagram, is shown below: 



Misdirected 
Q Effort 



Design 

Objective 

Clarity 



Conviction 



Development 

Time 



Design 

Problem 

Resolution 
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This diagram adds the conditions that Development Time decreases as 
Design Problem Resolution increases which in turn is driven by Conviction 
through Design Objective Clarity. The integrated diagram enhances the 
ability to compare future instances of Development Time with the 
accumulated knowledge by clearly and concisely displaying the current state 
of accumulated evidence and inferences. 

In comparing instances of Development Time from a second team at 
another company, using the Concept Engineering approach, an important 
difference was identified. This difference is exemplified by the following 
quotes: 

"Also, since we spent a lot of time with the requirement labels yesterday, 
perhaps we could shortcut a bit on the time without discussion and talk a 
little sooner." Process Facilitator 

"We should generate (requirement metrics) in pairs, then bring the result to a 
vote. Why not skip the voting step in pairs and vote as a group." Team 
Leader 

From these quotes a new category, Short-Cuts, can be derived. The 
second team, as a result of several disruptions in their project, planned to 
complete seven (of fifteen) steps of the Concept Engineering process in one 
week. Prior efforts, including the first team addressed above, allocated two to 
three weeks for these same activities. This caused the second team 
significant, self-imposed, time pressure. Time Pressure was also identified as 
a relevant variable relating to Development Time. A possible consequence of 
taking Short-Cuts can best be seen in one of the final comments during the 
second teams reflection period late Friday afternoon. 
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"Surprises me, that after all the discussion this week, some people don't 
know what others are talking about. I should say everyone doesn't know 
what the others are talking about." Development Engineer 

Adding the new categories, Short-Cuts and Time Pressure, to the diagram of 
accumulated knowledge above, results in the following diagram: 



Misdirected 




This causal-loop diagram shows two reinforcing loops (R1 and R2) and 
one balancing loop (B). The reinforcing loops imply that increases in Design 
Objective Clarity can decrease Development Time and subsequently Time 
Pressure as a result of less Misdirected Effort and/ or as a result of increased 
Conviction and Design Problem Resolution. The reduction in Time Pressure 
leads to decreased Short Cuts which increases Design Objective Clarity. The 
balancing loop implies that as Time Pressure increases Short Cuts also 
increase, thereby decreasing Time Pressure. However, Short Cuts also 
decrease Design Objective Clarity causing an increase in Misdirected Effort 
and a decrease in Conviction. This diagram can be continually validated as 
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additional instances of Development Time come to light; new variables will 
be added or relationships modified as dictated by the data. Eventually, 
modifications become fewer and a theory about Development Time, 
grounded in the data, can be clearly and concisely stated. 

Inductive System Diagram Reliability Assessment 

In the fall of 1992, seven Sloan School graduate students were 
presented with the Inductive System Diagram instructions and example 
presented above and extracts from my field notes, figure 3.2. The students 
ranged from Ph.D. candidates in System Dynamics to M.S. candidates with no 
prior exposure to System Dynamics. Each student independently prepared an 
Inductive System Diagram. In addition to providing final diagrams, many of 
the students also provided annotated transcripts, preliminary diagrams and 
the amount of time spent on the exercise. (Many of the individuals who 
indicated more time developed diagrams with fewer variables. I conclude 
from this, that some participants put more effort into the abstraction and 
simplification procedure described in step 5 of the Inductive System Diagram 
process. Therefore, a diagram with fewer variables and relationships may 
reflect a higher level of synthesis.) 
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The segments below come from an interview conducted with a member of a development team 
which was in the final stages of Concept Engineering. This team began in June 1992, spending an 
average of two days per week working together. This interview was conducted in September 
1992. The person being interviewed is the Engineering Manager for the business unit. 

^You've mentioned this was a process which was pretty well defined and handed to you, what is it 
about process that makes it appealing? 

Processes are nice just to know where you are. You need some sort of understanding to know 
where you are, where you want to be, are you heading in the right direction, sort of a navigational 
tool 

^Continue staying with process for a moment, what else is there? 

I think I still believe it is a good process, I see some potential benefits. It is going well, it is fun. I 
am almost tempted to say I am looking forward to it. It is one of the first times one of the TQM, 
activities has been enjoyable and fruitful. I haven't seen anything that indicates it is any less a 
potential solution to our problem. Although I have not had much contact with recent design 
projects, I don't believe we have ever gone out to customers without defining the product first. 

We haven't ever had so much discussion between engineers and marketing. We used to be 
sending a memo around. Engineering sent one. Marketing sent one, I don't think they read each 
others. I think all of this discussion ... 

iCan you describe what about the process makes you willing to spend so much time? 

I can say it the other way around, the negative, if I didn't think it was helpful I wouldn't spend so 
much time on it. 

iWhat is the Impact of interaction between marketing and engineering? 

I hope it is gonna makes things go a whole lot smoother down the road. Once we do decide what 
we are going to do there shouldn't be any surprises. Having spent all this time with the marketing 
guy, those decisions will be made pretty quick . We have covered so much it will be hard to think 
we haven't thought about something. This is a social process that probably hadn't existed before. 
This is surely going to break down some barriers. They are both going to know where the 
definitions came from and that will make the resolution of conflict a lot easier and there will also be 
a lot of day to day contact between them. Traditionally the designer, in the 6 to 9 months that the 
designer does his thing, normally doesn't talk to the marketing guy. When he is done he just gives 
it to them and they say "hey, where is this or that" 

^Why wont there be surprises? 

Communication issue. I hope the marketing and design guys will communicate after getting to 
know each other so well in this process. We have covered so many issues if something comes up 
they will say ya we covered that and will remember. Once we are done with this almost exhaustive 
idea generation. When we decide what to do we will understand a whole lot better what it is we 
are trying to do. It has been just recently that we have had a design document. Marketer writes the 
front page and the designer writes the back page and they don't read each others work. They are 
doing this together and will know where the roots are. 
iWhat is the impact of knowing the roots? 

They are going to know how we got to where we are in the product definition. I would hope that it 
will keep us from going back and going off in a tangent. Once we get to that definition there 
should not be a whole lot of waffling, we should feel pretty comfortable about it. I don't think 
there will be a whole lot of questioning. Other aspect if designers do come across something we 
didn't expect they will recall we talked about that and will have a basis for discussing this with the 
marketing guys. It should be a pretty quick way for reevaluating the definition. It is always nice 
too know where you have come through, need to know where you want to go but need to know 
where you've been and where you are. 
iWhat is the Impact of not waffling? 

the design should go a lot quicker. Designers wont get off on a tangent spending a week or two 
chasing something they think is neat, they will be more efficient. TTiey will know what to solve 
and what not to solve. 



figure 3.2 
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Reliability Assessment Method 

Each diagram was quickly reviewed for conformance with basic system 
dynamic modeling requirements. One diagram was rejected from further 
consideration as the author (someone with no system dynamics exposure) 
duplicated the same variable in multiple loops rather than connecting the loops 
through a single expression of the variable. The remaining six diagrams were 
reviewed to assess: the degree to which the diagrams reflect similar variables, the 
degree to which variables are connected in similar sequence; and the degree to 
which the overall structure is similar. 

Variable Comparison 

To assess the degree to which the diagrams reflected similar variables, each 
variable was written on a separate slip of paper. Those variables which expressed a 
similar concept were grouped together, figure 3.3. If a grouping contained multiple 
variable names from the same diagram, the original diagram was reviewed to 
ensure consolidation of variables was consistent with the original drawing. For 
example, in diagram #3, the variables: Speediness of Decisions, Development Time, 
Effectiveness, and Development Progress could be consolidated under the concept of 
Product Definition Time without changing the structure of the original causal loop 
diagram. However, in diagram #2, combining the variables. Time Spent in Process 
and Perceived Progress in Project under the Product Definition Time concept would 
have been inconsistent as the author of Diagram #2 linked the variable Time Spent 
in Process to the interview statement: "...if I didn’t think it was helpful I wouldn't 
spend so much time on it." After ensuring consistency, the group label was selected 
as the best exemplar of the concept in the grouping. Variables from each diagram 
which were not initially placed in a group were then reviewed to see if they could be 
added to an existing group, without changing diagram structure, to simplify 
analysis. 
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Figure 3.3 



* The author or Diagram 2 indicated that they wrote all variable names in a positive direction, i.e. the 
in-vivo codes of "missdirected effort" and "wasted effort" were abstracted into the category 
"effectiveness of effort" 
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Directed Arc Comparison 

To assess the degree to which the diagrams reflected similar pairwise 
associations , each diagram was first redrawn annotating which variables in 
the original diagram would be consolidated under the exemplar (bold) 
identified above in lieu of the original variable labels, e.g. figure 3.4. 
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Figure 3.4 



Each diagram was subsequently redrawn using only the common variable 
names, e.g., figure 3.5. In redrawing the original diagram with new variable 
names the sign of the arc connecting two variables may need to be changed. 
For example, the author of diagram 2 explicitly chose to write all variable 
names in a positive orientation, i.e., the references to "misdirected effort" and 
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"wasted effort" in the text were abstracted into the variable "Effectiveness of 
Effort". However, the exemplar chosen for this category was Missteps. As a 
result, the relationship between Design Clarity and Effectiveness of Effort is 
reversed from that between Design Clarity and Missteps. Appendix B shows 
the analysis of all diagrams. 




Using the redrawn diagrams, with common variable names, each 
pairwise directed arc connecting two variables was reviewed and the 
relationship annotated in figure 3.6. A two digit code was utilized for audit 
purposes; the first digit represents the directional relationship and the second 
digit represents the source diagram number. 
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Figure 3.6 

Reliability Assessment Results 

A review of figure 3.3 shows all six diagrams reference the following 
variables: Product Definition Time, Level of Cross Functional 
Communication, Design Clarity, and Use of Concept Engineering. 
Additionally, all diagrams except diagram 5 also referenced the variables 
Support for Process and Missteps. Four diagrams also referenced the variables 
Ease of Conflict Resolution and Shared Understanding. Unfortunately, due 
to the varying amounts of time spent in developing the diagrams and the 
differences in modeling experience, I am unable to conclude if the differences 
in variable inclusion represent failures on the part of the authors to identify 
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the concepts or are the result of more effort at abstraction and simplification. 
However, a review of figure 3.6 indicates that all variables which were 
connected by more than one author showed a consistent relationship. 

Stepping back from the detailed level of analysis to review the basic 
structures identified by the authors also shows a high degree of consistency. 
All six authors show a direct relationship from Use of Concept Engineering to 
Level of Cross Functional Communication. Five of the six authors show a 
direct relationship from Level of Cross Functional Communication to Design 
Clarity and the sixth author shows an inverse relationship to Product 
Definition Time. Furthermore, four of the remaining five authors map a 
inverse relationship from Design Clarity to Product Definition Time usually 
via the intervening variable Missteps. All five authors who show a 
relationship from Product Definition Time indicate it has an inverse 
relationship either directly to the Use of Concept Engineering (1 diagram) or 
indirectly through the variable Support for Process (4 diagrams). In summary, 
all participants in the study identified the same basic structure, figure 3.7. 
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Figure 3.7 



68 



Increased Use of Concept Engineering causes increased Levels of Cross 
Functional Communication which results in increased Design Clarity which 
leads (via reduced Missteps) to a reduction in Product Definition Time which 
in turns leads to an increase (via Support for Process) in the Use of Concept 
Engineering. Furthermore, it should be noted that each variable generated 
from the data can be operationalized and the predicted cause and effect 
relationships tested. 

The results of this preliminary assessment of Inductive System 
Diagrams indicates that they appear to be reliable with respect to variable 
indentification and integration. However, a more complete test, involving 
more subjects and evaluators is required before a more definitive statement 
of reliability can be made. 

Causal-loop Diagram Limitations 

Causal-loop diagrams do not show the level and rate substructure of 
the system (Goodman 1974, Morecroft 1982, Richardson 1986). In cases 
involving rate-to-level pairwise directed arcs, the traditional definitions of 
positive and negative polarity fail because accumulation effects are lost 
(Richardson 1986). In the filling of the beer glass example used previously in 
this chapter, the link from the rate of beer flow to the level of beer in the glass 
fails the traditional definition; here a decrease in the rate of flow from the tap 
will not produce a decrease in the level of beer in the glass (Richardson 1986; 
p.l60). As a result, accurate prediction of system behavior is difficult using 
only causal-loop diagrams and more detailed flow diagrams are required 
before developing simulation models (Goodman 1974, Morecroft 1982, 
Richardson 1986). 
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Wolstenholme (1982; p.547) makes a clear distinction between the 
system description (qualitative) analysis aspects of system dynamics and the 
simulation modeling (quantitative) techniques and states: "a good system 
diagram can formalize and communicate a modeler's mental image and 
hence understanding of a given situation in a way that the written language 
cannot." Coyle (1983; p.885) states that the difficult part of the operations 
research discipline is to clearly describe the interrelationships of the system 
under investigation and that system diagrams require "not much more than 
patience and persistence to apply ... in reaching a good first approximation to 
an adequate breadth of view in considering a complex problem." Goodman 
(1974) concludes that while causal-loop diagrams are insufficient for 
constructing simulation models they are useful for model conceptualization 
by organizing principal components and feedback loops. 

Conclusion 

Inductive System Diagrams have been introduced as a diagram-based 
method for systematic field-based hypothesis development and integration. 
Inductive System Diagrams build on the strengths of accepted coding practices 
for variable development. They can be used to integrate variable 
relationships and are easily modifiable as additional information becomes 
available. As a result, they facilitate the ability of researchers to use the 
constant comparative method of analysis, an accepted approach for theory 
generation. The Inductive System Diagram method was found to have 
reliability in a small scale experiment involving experienced and novice 
dynamic model builders. Additionally, they allow for theory validity testing 
against the criteria of: verifiable data, explicit inferences and disconfirmable 
predictions. 
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Chapter 4: Time to Market Dynamics 



The expression "Time to Market" is composed of two distinct components, 
time and market.^ In this comparative study, a fundamental difference was 
observed in the product concept development teams depending on whether they 
focused on TIME or MARKET in the expression Time to Market. After more than 
a year and a half of field observations, interviews, and analysis, key variables 
associated with the product concept development decision process and Time to 
Market dynamics have been identified using the inductive system diagram 
process, described in the previous chapter. In this chapter, I present an 
investigation of the causes, consequences and constraints associated with a focus 
on TIME or MARKET in the product concept development decision process. 

TIME to Market Orientation 

Decreased TIME to market has been identified as a key ingredient in 
successful new product development (Mansfield 1988, Takeuchi & Nonaka 1986, 
Gupta & Wilemon 1990). There are significant market share benefits to early 
market entrants (Urban et al. 1986) and considerable penalties for being late to 
market. For example, McKinsey and Company claims shipping a product six 
months late can reduce life cycle profits by one third in high growth, short life 
cycle markets (Reinertsen 1983). Additionally, competitive pressures are 
reducing product life cycles, further increasing the pressure to reduce product 
development time (Mansfield 1988, Schmenner 1988, von Braun 1990). 

Gold (1987) suggests several strategies to accelerate product and process 
development. These can be categorized as increasing reliance on external sources 
of innovation, increasing reliance on internal development of innovation, and 

’David Walden of BBN Inc. first pointed out this important distinction to me. 
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increasing reliance on innovative management approaches. Millson, Raj and 
Wilemon (1992) outline a general framework for reducing development cycle time 
that consists of simplification, delay elimination, step elimination, operation 
speedup and parallel processing. Mansfield (1988) shows that innovation time can 
be reduced by increasing innovation cost. Reinertsen (1983) states that in both high 
and low growth markets, over-running development costs 50% to meet schedule 
had only a 4% impact on life cycle profit before tax. Bower and Hout (1988) state 
that fast cycle capability is a management paradigm which shapes an organization’s 
systems and attitudes around the simple concept that time is money. 

Based on the observations and analysis of my field research and 
supported by the above discussion, I propose the following proposition: 

PI: A TIME to market orientation increases pressure for progress. 

Time to MARKET Orientation 

Considerable research on new product development success highlights 
the central importance of understanding user needs, i.e., the market (see for 
example: Rothwell et al. 1974, Cooper & Kleinschmidt 1986, Pavia 1991). 

Houston (1986) states that customer focus, profits and organizational integration 
are frequently associated with the marketing concept and have become 
synonymous with having a customer orientation. Shapiro (1988) describes the 
characteristics of the market driven company to include widespread 
dissemination of important buying influence information, interfunctional 
decision making, and committed coordinated decisions. Narver and Slater (1990) 
state that marketing orientation consists of three behavioral components: 
customer orientation, competitor orientation, and interfunctional coordination. 
Kohli and Jaworski (1990) in an extensive review of the literature found three 
core themes related to market orientation: customer focus, coordinated 
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marketing and profitability. However, the results of their 62 field interviews, 
conducted with a diverse cross section of managers, found that managers felt 
profitability was a consequence not a condition of market orientation. 

Based on the observations and analysis of my field research and 
supported by the above discussion, I propose the following proposition: 

P2: A time to MARKET orientation increases customer oriented bias and 

functional integration. 

Comparison of Product Concept Development Teams 

A TIME to market team is one which attempts to specify the design 
objectives in an accelerated period of time. The team is under a great deal of 
pressure for progress and displays a willingness to make decisions with 
recognized data deficiencies in order to meet the (usually aggressive) development 
schedule. Participants orient their analysis of the issues to support their, often 
preconceived, perspective of the product concept. In the development team 
meetings, partisan behavior, in which individuals stake out positions and 
vigorously defend them, dominates. The engineers discuss product attributes 
from the perspective of technology opportunities and constraints. Marketers 
discuss product attributes with respect to market segments and competitors. 
Although both groups are at the same meeting, they don't participate in the same 
process: the language used is different, the relative emphasis on product attributes 
is often different, and individuals can be observed to periodically disengage from 
the decision process based on discussion subject matter. Product concept decisions 
are ultimately made, but it is difficult for the entire team to recreate and defend the 
decision choices to the management review board. When all is said and done, one 
or more groups lack commitment to the product concept and there is a high 
expectation that the final product will differ from the initial concept. 
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A time to MARKET team is one which attempts to develop credible design 
objectives which reflect a deep appreciation of the customer's requirements. The 
team is characterized by decision analysis oriented to maximize customer benefit. 

In development team meetings, every individual participates in all aspects of the 
decision process. Members frequently put their statements in the context of specific 
customer encounters to clarify or emphasize their positions. Relevant issues and 
information regarding design objectives are considered to everyone's satisfaction 
before the team moves on to subsequent development activities. This cross- 
functional collaboration to create a common appreciation of the design objectives is 
apparent when the team presents the product concept to the management review 
board. All team members display a commitment to the product concept and can 
credibly trace their decision process when required to justify their choices. 

The observed variable relationships are outlined in the table below. 





TIME to market 


time to MARKET 




orientation 


orientation 


Decision Variables 






Pressure for Progress 


Higher 


Lower 


Systematic Concept Analysis 






Prejudiced Perspective 


Higher 


Lower 


Functional Integration 


Lower 


Higher 


Analysis Depth 


Lower 


Higher 


Objective Function 






Supporting Evidence 






Contextual Awareness 


Lower 


Higher 


Process Participation 


Lower 


Higher 


Traceability 


Lower 


Higher 


Design Objective Appreciation 






Requirement Clarity 


Lower 


Higher 


Requirement Credibility 


Lower 


Fligher 


Substantive Progress 






Concept Commitment 


Lower 


Higher 


Misdirected Effort 


Higher 


Lower 


Constraints 






Labor-hour Requirement 


Higher 


Lower 
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The dynamics of a TIME versus MARKET orientation in the expression Time- 
to-Market may be easier to understand by representing the data in the table above as a 
high level inductive system diagram (see Chapter 3). 



TIME 



vs. 




A relative emphasis on TIME increases Pressure for Progress and reduces the 
opportunity for Systematic Concept Analysis. This reduction in systematic analysis 
decreases the labor requirement and consequently the concept development time. 
However, it also decreases the Supporting Evidence needed to justify concept 
decision choices. The resulting reduction in Design Objective Appreciation 
subsequently reduces Substantive Accomplishments as time and resources are spent 
on tangents and detours in downstream development efforts. The net result is 
increased Development Time and increased Pressure for Progress. 

A MARKET orientation decreases Pressure for Progress, relative to the TIME 
oriented development teams, and increases Systematic Concept Analysis. The 
increase in Systematic Concept Analysis increases Supporting Evidence, but also 
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increases the labor requirement and Concept Development Time. However, the 
resulting increase in Design Objective Appreciation focuses development efforts 
thereby increasing Substantive Accomplishments which in turn will decrease 
Development Time and Pressure for Progress. 

The dynamics described above represent a classic "Fixes that Fail" archetype 
(Senge 1990) in which the unintended consequence of a problem solution over time 
contributes to the problem it was trying to solve. In this case the emphasis on 
reducing TIME to market decreases concept development time but inadvertentdly 
reduces design objective appreciation resulting in waste and rework in downstream 
development activities increasing total development time. On the other hand, the 
fundamental solution, an emphasis on MARKET, actually reduces total time by 
saving the time spent on misdirected downstream development efforts. 

Presentation Structure 

The presentation of the underlying variables identified in this study follows 
the order of the left-hand column of the concept decision variable table in the 
previous section. (This order is also a clockwise rotation around the inductive 
system diagram presented above.) The decision variables observed in the concept 
development process include pressure for progress and systematic concept 
development (prejudiced perspective, functional integration, and analysis depth). 
The objective function of product concept development activities was observed to be 
the maximization of supp>orting evidence (contextual awareness, process 
participation, and traceability), design objective appreciation (requirement clarity 
and credibility) and substantive development accomplishments (commitment and 
misdirected effort). The primary constraint observed was available resources, 
specifically labor-hours. The table below provides a brief definition of each variable 
and indicates the page which contains an expanded variable description. The 
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expanded description includes evidence in the form of representative quotes, a 
diagram of the inference from the evidence and a propositional statement of the 
prediction from the inference. Following the expanded definitions, the variables are 
integrated into an inductive system diagram. A discussion of conclusions, 
contingencies and rival plausible hypotheses follows the integrated inductive system 
diagram. 



Variable Name 


Page 


Definition 


Pressure for Progress 


8 


An implicit or explicit force on the 
development team to proceed rapidly 
through development activities. 


Prejudiced Perspective 


10 


The formation of an opinion on the product 
concept without knowing all the facts. 


Functional Integration 


11 


The degree of interaction between functional 
groups, primarily marketing and engineering. 


Analysis Depth 


13 


The degree of investigative thoroughness 
associated with concept development 
decisions. 


Contextual Awareness 


15 


The ability of a development team member to 
place a requirement statement in the context 
of the customer’s environment. 


Process Participation 


16 


The active involvement of participants in the 
complete (requirement identification, idea 
development, and concept selection) process. 


Traceability 


17 


The capability to recreate decision choices 
with the supporting justification. 


Requirement Clarity 


18 


A product definition in which the vital few 
requirements and their relative priorities are 
identified and agreed upon. 


Credibility 


20 


The perceived validity and accuracy of 
information related to design objectives. 


Concept Commitment 


21 


The level of support the concept has earned 
from the development team and their 
managers. 


Misdirected Effort 


22 


Development activities resulting in detours, 
tangents and missteps relative to the design 
objectives. 


Labor Requirement Gap 


23 


The difference between required and 
available labor for concept development. 
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Decision Variables 

In this study the decision variables observed in the concept development 
problem include pressure for progress and systematic concept development. 
Pressure for progress was either an explicit or implicit force on the development 
team to rapidly proceed through development activities. Systematic concept 
development consists of three components: prejudiced perspective, functional 
integration, and analysis depth. Prejudiced Perspective indicates the formation 
of an opinion on the product concept without knowing all the facts; it was 
observed as biased decision making by internal stakeholders. Functional 
Integration reflects the degree of interaction between various functional groups 
in the stages of concept development; cross functional interaction was observed 
to range from collaborative to partisan. Analysis depth represents the degree of 
overall thoroughness of analysis associated with the concept development 
process; investigations were observed to vary from comprehensive to 
incomplete. It was observed that each of these variables was capable of being 
influenced by management to a greater or lesser, but always non-trivial, extent. 

Pressure for Progress 

Several researchers indicate that the pressure for accelerated new product 
development can cause development organizations to conduct a less than 
thorough job in order to have the appearance of progress (Van de Ven 1986, 
Gupta & Wilemon 1990). Contributing to this phenomenon may be the 
disconnect between product development theory and practice. Bower and Hout 
(1988) specifically emphasize the requirement that fast cycle companies have 
development processes which are well defined and understood. They state that 
in addition to visibility of the process from start to finish, the fast cycle 
organization understands the interrelationships of the process components and 
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organizational policies. The strategies outlined by Millson, et al. (1992), e.g. step 
elimination, delay elimination and parallel processing, all require development 
process understanding. However, the product concept decision process is not 
well defined or understood. The observed disconnect between the requirement 
for process understanding in theory and the lack of process knowledge in 
practice is consistent with other studies of product development theory and 
practice which indicate that what the literature recommends and what the 
actually happens are significantly different (Cooper & Kleinschmidt 1986, Gupta 
& Wilemon 1990, Mahajan & Wind 1992). 

In this study. Pressure for Progress was observed to lead to decision 
process speedup. When pressured for progress participants were observed to 
display self-interest oriented behavior based on a prejudiced perspective. 
Pressure for Progress was also observed to lead to a willingness to make 
decisions with data deficiencies. Pressure for Progress was observed to dominate 
other decision variables specifically causing stakeholder orientation and 
incomplete analysis. 

Evidence, inferences and propositions are provided below. 

"Some of the features that make the applications engineer job easier, 1 wouldn 't have 
thought about this if I was under pressure; I wouldn 't have put them in my design." - 
design engineer 



Pressure Stakeholder 

for ► Prejudiced 

Progress ^ Perspectives 

P3: As Pressure for Progress increases Prejudiced Perspectives increase. 
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"You need to be very quick to get something up on the screen. In the initial stages you 
don 't want to get all hung up on all of the goals" - design engineer 



Pressure 

for 

Progress 



S Analysis 



^ Incomplete 



P4: As Pressure for Progress increases Analysis Depth decreases. 
Prejudiced Perspectives 

Van de Ven (1986) claims the first, of four, central problems in managing 
innovation involves the problem of managing human attention; overcoming 
individuals', and organizations', natural tendency to be focused on and 
protective of, existing practices. The self-interested behavior of individuals (or 
groups) in settings which require interdependencies can lead to conflict (Kohli & 
Jaworski 1990, Ancona & Caldwell 1992). Self-interested behavior and conflict 
lead to reduced cross-functional integration (Gupta et al. 1986, Souder 1988, 

Kohli & Jaworski 1990, Ancona & Caldwell 1992). 

Dougherty (1992) suggests that viewing the product from the user's 
perspective can provide a basis for developing a common understanding of the 
desired innovation. Developing the user's perspective involves more than 
obtaining the customer's verbalized needs and preferences; it includes analysis of 
the factors which influence those needs and preferences (Kohli & Jaworski 1990). 
Placing design engineers in direct contact with customers provides an 
opportunity for developing this deeper level of understanding (Rothwell et al. 
1974, Kohli & Jaworski 1990, Bailetti & Guild 1991). 

In this study, analysis was observed to occur from either a customer or 
stakeholder perspective. Customer oriented perspective is present when the 
concept development decision process biases innovation efforts towards 
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customer benefits. Customer orientation was most often evident by the use of 
numerous, specific, references to customers during all phases of the design 
objective decision process. In contrast to customer orientation is stakeholder 
orientation. Stakeholder oriented perspective is evident when the decision 
process is biased towards satisfying the prejudiced opinions of people in 
positions of power (on the team or in management) who often dictate product 
design objectives. In this study, it was observed that Customer Orientation 
increased Contextual Awareness and Crossfunctional Collaboration. 

Evidence, inferences and propositions are provided below. 

"Getting oriented to customer needs, in some cases changing biases and getting 
intimately familiar with customer requirements and environments." - team leader 

Prejudiced ^ Contextual 

Perspectives o Awareness 

P5: As Prejudiced Perspectives decrease Contextual Awareness increases. 

"If development and marketing hear more of the same stuff from customers it has to build 
a better working relationship." - team leader 

Prejudiced ^ Crossfunctional 

Perspectives O Collaboration 

P6: As Prejudiced Perspectives decrease Functional Integration increases. 
Fimctional Integration 

Lawrence & Lorsh (1967; p.ll) define integration as the "quality of the 
state of collaboration that exists among departments that are required to achieve 
unity of effort by the demands of the environment". Functional integration has 
been shown to be a key factor in successful innovation (Rothwell et al. 1974, 
Gupta et al. 1986, Gupta & Wilemon 1988, Pinto & Pinto 1990, Moenaert & 
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Souder 1990, Dougherty 1992, Song & Parry 1992). Innovation is essentially an 
information based activity and as a result information transfer is the major 
integration vehicle (Moenaert & Souder 1990). Communication effectiveness has 
also been linked with innovation success (Rothwell et al. 1974, Ancona & 

Caldwell 1992). However, achieving effective communication and integration is 
difficult; nearly 2/3 of the 289 new product development efforts studied by 
Souder (1988) experienced some degree of interface disharmony between R&D 
and marketing. 

In this study, all of the companies observed had a practice of assigning 
multiple functions, marketing and engineering at a minimum, to product concept 
development teams. However, the degree of interaction among the various 
functional groups on the development team was observed to range from 
collaborative to partisanship. Crossfunctional collaboration is the ability of 
different functional groups to work together in the concept development decision 
process. In collaboration, a synthesis of the perspectives of the functional 
representatives assigned to the team is incorporated into the concept development 
decision process. In contrast to crossfunctional collaboration is functional 
partisanship: the advocation or strong support of one particular perspective. In 
this study, crossfunctional collaboration led to process participation. Functional 
Partisanship was observed to lead to incomplete analysis. 

Evidence, inference and propositions are provided below. 

"If designers do come across something we didn 't expect they will recall we talked about 
that and will have a basis for discussing this with the marketing guys. It should be a 
pretty quick way for reevaluating the product definition." - engineering manager 

Crossfunctional . Process 

Collaboration Participation 

P7: As Functional Integration increases Process Participation increases. 
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"The designer and marketing guy go to the field and get information. Interpretation of 
the data and how to apply it to a product is left up to the individual and people interpret 
differently based on their backgrounds" ~ design engineer 

Functional ^ Incomplete 

Partisanship s Analysis 

P8: As Functional Integration decreases Analysis Depth decreases. 
Analysis Depth 

Several studies indicate that innovation success is significantly impacted 
by the number of product development process steps completed; the more 
thorough the job the more likely the success (Cooper & Kleinschmidt 1986, Gupta 
& Wilemon 1990, Wilson 1990, Mahajan & Wind 1992). Unfortunately, Cooper 
and Kleinschmidt (1986) found that "up front" new product development 
activities were more likely to be performed poorly or not at all. Additionally, 
Moenaert and Souder (1990) indicate that industrial new product development 
success is related to the effectiveness of information processing. However, Gupta 
& Wilemon (1990) found 47% of the participants in their study were frustrated by 
the lack of attention to detail during product development. 

In this study, the analysis associated with product concept development 
was observed to range from comprehensive to incomplete. Comprehensive 
analysis entails a thorough identification of key design requirements, an 
extensive development of alternative ideas and a systematic concept selection 
process. In contrast to comprehensive analysis is incomplete analysis. 
Incomplete analysis is characterized by decisions with both recognized and 
unrecognized data deficiencies and resulting premature switching between 
decision process stages. In this study, it was observed that Comprehensive 
Analysis leads to traceability whereas Incomplete Analysis does not. 
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Evidence, inference and propositions are provided below: 

Marketing Rep: "Lets move on to the competition comparison sheet; did you get .1%? 
Engineering Rep: "No I didn't; but of course I'd be better." 

Ma rket ing Rep: "A ny idea 7 " 

Engineering Rep: "Totally a guess; maybe 10 instead of 15." 

Marketing Rep: "I'll leave it blank. " 

Observer: Product definition presentation to management one week later listed 10 for this 
specification. 

"When I use 'systematic thinking' in this I think here is what the customer said and here 
is the requirement which matches what he said." - marketing manager 



"oeX ^ TraceabUity 

P9: As Analysis Depth increases Traceability increases. 



Objective Function — Supporting Evidence 

Supporting Evidence provides participants the ability to justify the 
decisions made during the entire concept development decision process. There 
were three observed conditions associated with supporting evidence: contextual 
awareness, process participation and decision traceability. In the time to 
MARKET teams, references to customer contexts were used to clarify issues 
throughout the entire concept development decision process. Participation in the 
decision process allows the design team member to know the background on 
how the design objective decisions were reached. Evidence provides participants 
a way to confidently recreate the decision process should/ when the need arise in 
the future. 
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Contextual Awareness 

Moenaert and Souder (1990; p.221) state; "A variable that has been seldom 
addressed in previous research, and which seems to be essential to the use of 
information received from other functional fields, concerns the contextuality of 
the received information. The contextuality of information refers to the degree to 
which the source has given the receiver the necessary information and references 
such that the receiver can see the relevance of this information for his/her work 
on a particular project." They conclude that extrafunctional information, without 
supporting context, cannot be processed and used. 

In this study. Contextual Awareness of the customer's requirements was 
observed to increase the clarity of the design objective. Contextual awareness is the 
ability of development team members to place a requirement statement in the 
context of the customer's environment. Written requirement statements ideally 
represent a high fidelity translation of the customer's actual needs. However, even 
in the best processes for capturing the voice of the customer, the written requirement 
statement lacks the affective qualities of an actual customer interaction and is subject 
to different interpretations. Placing requirement statements in the context of the 
customer's use environment clarifies the intent of the requirement statement. In this 
study, it was observed that stories of real customer experiences, whether obtained 
specifically in efforts associated with the current project or in other efforts, were 
used by development team members to clarify written requirement statements. 

Evidence, inferences and propositions are provided below. 

"The team began a discussion of what a requirement statement meant. Eventually they 
went back to the interview transcript and rewrote the requirement statement." - observer 

Contextual ^ Requirement 

Awareness s Clarity 

PIO: As Contextual Awareness increases Requirement Clarity increases. 
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Process Participation 

Several studies indicate that interaction between producers and users of 
market information significantly impacts the credibility and utilization of the 
information in the innovation process (Deshpande & Zaltman 1982, Deshpande & 
Zaltman 1984, John & Martin 1984). Gupta and Wilemon (1988) state that credibility 
has two main components — information credibility and source credibility. 
Deshpande and Zaltman (1982) conclude that personal interaction increases trust in 
the source and consequently the content of the research. Additionally, 
crossfunctional participation in market research increases the effectiveness of 
information exchange between functions (Deshpande «Sc Zaltman 1984, Kohli & 
Jaworski 1990). John and Martin (1984) found that the proximity of participants to 
the marketing planning activities enhanced credibility which they attribute to 
communication facilitation. 

In this study. Process Participation was observed to build design objective 
credibility. Process participation represents the active invoivement of participants in 
the complete decision process. This implies that individuals participate in 
requirement identification, idea development and concept selection activities. This 
is contrasted with event participation in which individuals participate in some 
events and not others, i.e. participation in concept selection but not requirement 
identification activities. Participation in all stages of the design objective decision 
process provided team members with a common understanding of the events which 
led to the concept selection; this increases requirement clarity and credibility. 

Evidence, inferences and propositions are provided below. 
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"They (marketing and engineering) are doing this together and will know where the roots 
are. They are going to know how we got to where we are in the product definition. " - 
engineering manager 



Process 

Participation 



S 



Requirement 

Clarity 



Pll: As Process Participation increases Requirement Clarity increases. 



"There was engineers in the process. It wasn 't marketing pukes saying this is what it is. 
When we got into the room there was built in credibility because the person who is 
processing the information is witnessing data collection." - marketing manager 



Participation § ^ Credibility 

P12: As Process Participation increases Credibility increases. 



Traceability 

Van de Ven (1986) states that the legitimacy of the decision process is the 
dominant evaluation criteria used to assess innovative ideas as the ideas 
themselves can rarely be judged. Deshpande and Zaltman (1982) found that 
managers were inclined to pay critical attention to research methodology when 
the findings were surprising. Moenaert and Souder (1990) found that 
information which was not formally substantiated by convincing evidence was 
less likely to be used. They also observe that a disadvantage of face-to-face 
discussions is the absence of hard copy which can be used in the future to justify 
actions taken. 

In this study. Traceability was observed as a highly desirable outcome for 
justifying the decisions made during the product concept development process. 
Traceability includes, but is not limited to, documentation of the outcomes of the 
decision process. Just as importantly, traceability regarding the concept 
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development decision process itself provides credibility to the identification, 
development and selection of design objectives. In this study, it was observed 
that the ability to document the decision process increased the credibility of the 
design objective decisions. 

Evidence, inferences and propositions are provided below. 

"/ have higher confidence and better ability to sell to executive committee because the 
process is self -documenting." - marketing manager 



Traceability 



^ ► Credibility 



P13: As Traceability increases Credibility increases. 



Objective Function — Design Objective Appreciation 

Design objective appreciation occurs when the development team has 
discriminating perception and confidence in a clearly defined set of 
requirements. Requirement clarity results from understanding the vital few 
requirements and the relative priorities within this set of requirements. Design 
activities fundamentally involve making tradeoffs; design objective appreciation 
facilitates tradeoff optimization. The development team needs to clearly 
understand the subset of the total universe of requirements which will be 
emphasized in the product concept to ensure effort is focused effectively. 
Requirements are assessed for credibility, by the development team, in order for 
them to have confidence in the relative merit of the tradeoff decisions. 



Requirement Clarity 

Understanding user needs has long been recognized as a significant factor 
to new product development success (Roth well et al. 1974, Cooper & 
Kleinschmidt 1986). The absence of a clear product definition has been linked to 
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instability in product and marketing plans (Gupta & Wilemon 1990, Wilson 
1990). Gupta and Wilemon (1990) found poor definition of product requirements 
was the reason most cited for product development delays. 

In this study. Requirement Clarity was observed to consist of two 
components: the identification and agreement on the vital few requirements and 
their relative priorities. The establishment of the vital few requirements focused 
development efforts. In all observed projects, the development team attempted 
to identify a small subset of the total requirements which would differentiate the 
proposed product from the competition (either internal or external). Knowing 
the relative importance of these key requirements assists in making development 
tradeoffs. The relative requirement priorities clarify what to work on and, just as 
importantly, what not to work on. There was a concerted effort on each observed 
development team to clearly establish the relative priorities of acknowledged 
design objectives. The product definition, by focusing on only the vital few 
requirements, is designed to serve as a road map for the development team, 
providing direction and flexibility for making tradeoffs while minimizing 
unproductive effort. 

Evidence, inferences and propositions are provided below: 

"Documented lists of the most important things to be concerned with ... all too often you 
put blinders on, getting in a rut, attacking one piece of the problem and let other aspects 
slide. " - design engineer 

"The implication of having visibility (of customer requirement priorities) is it tells you 
what is good enough, where you have to put efforts or where you can compromise, not 
spend time . " - design engineer 



Requirement 

Clarity 




Misdirected 

)evelopment 

Effort 



P14: As requirement clarity increases misdirected development effort 
decreases. 
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Credibility 

John and Martin (1984) define credibility as a composite of six components 
related to the perceived quality of the marketing plan; these components assess 
whether the plan is: realistic, accurate, specific, consistent, complete, and valid. 
Gupta and Wilemon (1988) conclude that in organizations with a high degree of 
integration between marketing and R&D, R&D personnel perceive marketing 
information to be: realistic and valid, objective, consistent and complete, useful 
and appealing. Moenaert and Souder (1990) define credibility as a measure of 
the degree to which the receiver believes the information to be undistorted and 
state that the credibility of received information will be a positive function of: 
validity, accuracy and clarity and a negative function of surprise. Deshpande & 
Zaltman (1982) focus less on "credibility" per se but rather on the instrumental 
use of knowledge; it is the knowledge applied to a particular decision. They 
identify four dimensions of instrumental use: information relevance, information 
surplus, recommendation implementation and overall satisfaction. 

In this study, it was observed that credible design objectives obtain 
commitment. Requirement statements were tested, either formally or informally, 
for credibility. The credibility of the vital few requirement statements reinforces 
commitment to the design objective. Requirements without credibility are 
discounted, thereby reducing commitment to the stated design objectives. 

Evidence, inferences and propositions are provided below. 

"Post program approval then run in automatic; hard work but people already know what 
to do and want to do it." - engineering manager 



Credibility 



S 



Committment 



PI 5: As credibility increases commitment increases. 
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Objective Function — Substantive Development Accomplishment 

Substantive development accomplishments are development actions 
which lead directly to real progress towards realizing the design objectives. 
Substantive development accomplishment, in the context of the concept 
development decision process, has two dimensions: concept commitment, and 
focused effort. Concept commitment represents the ability of a product concept 
to garner enough enthusiasm and support from the development team that it 
does not change during subsequent development activities. Focused effort 
describes a development process which is direct compared to one filled with 
detours. 

Concept Commitment 

Several authors claim commitment is a result of participation in the 
formulation stage of a project (Gupta, Raj & Wilemon 1986, Shapiro 1988, 

Moenaert & Souder 1990, Gupta & Wilemon 1990, Bailetti & Guild 1991). Gupta 
& Wilemon (1990) indicate that a lack of commitment is related to changes in 
product definition and low management support. 

In this study it was observed that design objective appreciation led to concept 
commitment which in turn led to reduced misdirected development effort. Concept 
commitment represents the level of support the product concept has earned from 
both the development team and their managers. Committed individuals ensure the 
necessary work get accomplished. Committed managers provide the necessary 
resources to support the team's efforts. Concepts with commitment don't change 
during the development process. Changing product concepts often make prior 
development activities useless creating waste and rework. 

Evidence, inferences and propositions are provided below. 
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"]Ne didn 't have confidence, (therefore) we didn 't want to tinker because it would be 
wasted effort.” - engineering manager 

Concept ^ Misdirected 

Commitment o Effort 

P16: As concept commitment increases misdirected effort decreases. 
Misdirected Development Effort 

As mentioned previously, the absence of a clear product definition has 
been linked to instability in product and marketing plans (Gupta & Wilemon 
1990, Wilson 1990). Instability can manifest itself in significant changes in 
direction or in "creeping elegance" (Gupta and Wilemon 1990). Wilson (1990) 
found that product definition instability could lead to more staffing, funding and 
time. 

The System Dynamics literature has investigated the impact of project 
changes in a variety of development settings, e.g. construction (Homer et al. 199), 
research and development (Roberts 1964, 1978, Richardson & Pugh 1981), 
shipbuilding (Cooper 1980, Reichelt & Sterman 1990), and software (Abdel- 
Hamid & Madnick 1989, 1991), and consistently find project changes directly and 
indirectly increase development time and costs. Abdel-Hamid and Madnick 
(1989; p.l427) state: "the rework necessary to correct such software errors 
obviously diverts the project team’s effort from making progress on new project 
tasks, and thus can have a significant impact on the projects progress rate." They 
explicitly model the relationship from progress-rate to forecast completion date 
to schedule pressure. Roberts' discussion of research and development project 
control indicates that "there is no intrinsically correct measure either of 
engineering effectiveness or of problems solved or of the task left to be done.... the 
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obvious concrete and measurable variables are often basically unrelated to the 
amount of effort required to get the job done" (1964; p.l69). Roberts indicates 
(1964, 1978) that one result of this measurement difficulty is a delayed response 
to events which impact the development schedule. 

In this study, it was observed that a reduction in Misdirected 
Development Effort decreased development time. In addition to tangible time 
savings, reduced misdirected effort was observed to increase the sense of 
accomplishment of the team. It is assumed that the time saved through 
reductions in tangents, detours, missteps, etc. decreases Pressure for Progress 
and Labor-hour Requirements. 

Evidence, inferences and propositions are provided below. 

"This process would have provided a clearer vision, a straighter path to the end result...! 
see the process saving time by eliminating missteps" - design engineer 

Misdirected ^ Development 

Effort s ^ 

PI 7: A decrease in misdirected effort decreases development time. 

P18: A decrease in development time decreases pressure for progress. 

P19: A decrease in development time decreases labor-hour requirements. 

Constraints — Labor-hours 

Constraints are limits placed on decision variables. Although the list of 
conceivable constraints which can be imposed on the decision variables outlined 
above is considerable, one, required labor-hours, dominated the observations in 
this study of product concept development. As the innovation process is primarily 
informational (Moenaert & Souder 1990) it can be argued that mental capacity 
(labor) is the critical resource. The Labor-hour requirement gap represents the 
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difference between required laborhours and available laborhours. When Labor- 
hour requirements exceed availability the gap is large. In this study, labor 
availability was fixed by the team size. In every observed development team, the 
membership remainded the same or was reduced during the roughly six month 
observation period. The labor requirement can be driven by the project itself or by 
other projects team members participate in. In this study, it was observed that 
when a Labor-hour requirement gap exists there is an increase in concept 
development time. It was also observed that systematic concept development 
(comprehensive, collaborative, customer oriented analysis) increased the labor 
requirement. 

Evidence, inferences and propositions are provided below. 

"This process (CE) is taking a long time; not just a long elapsed time because it is not 
calendar time. But in terms of people time it is extensive." - marketing manager 



Analysis 

Depth 




Labor 

Requirement 

Gap 



P20: As Analysis Depth increases labor-hour requirements increase. 

"We have sales quotas to meet, sales growth rates to meet, as a result the managers won 't 
give time to gather and analyze the information (for products not yet defined)." - manager 



Labor 

Requirement 

Gap 



S 



Concept 

Development 

Time 



P21: As the labor requirement gap increases concept development time 
increases. 
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Integration 

Combining the relationships identified above into an integrated diagram 
allows us to better understand the interactions between the variables associated 
with the product concept development decision. Each arc of the diagram has 
been annotated with the relevant proposition number. 




This TIME vs. MARKET inductive system diagram can describe a vicious 
or virtuous cycle of product concept development depending on which decision 
variables are emphasized. A vicious cycle begins when pressure for progress 
leads to incomplete concept analysis and stakeholder prejudiced perspectives in 
decision analysis. The resulting decrease in functional integration further 
degrades analysis depth and reduces participation of all team members in the 
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concept decision process. The resulting lack of requirement clarity and 
credibility leads to low concept commitment and misdirected development 
effort. Ultimately, the waste and rework increases development time further 
increasing pressure for progress. 

The TIME vs. MARKET inductive system diagram can also describe a 
virtuous cycle in which an increase in customer orientation perspectives lead to 
an increase in Functional Integration and Contextual Awareness. As a result, a 
more thorough analysis, grounded in the context of the customer's environment, 
is conducted with the active participation of all development team members. 

This common appreciation of the concept decision process and outcomes leads to 
a higher degree or requirement clarity and credibility. In turn, commitment to 
the product concept is higher and misdirected development effort is reduced. 
Ultimately, development time is reduced, thus decreasing pressure for progress 
and labor requirements. 

The dynamics associated with either a TIME or MARKET emphasis in the 
expression time to market can be explained by the inductive system diagram 
above. The level of detail displayed in the diagram provides a more 
comprehensive causal map than currently exists in the literature. Additionally, it 
identifies the dysfunctional and unintended consequence which results from a 
TIME to market orientation during the product concept decision process. 

Plausible Rival Hypotheses 

The inferences and propositions integrated into the TIME vs. MARKET 
inductive system diagram above are the result of a comparative analysis of 
product concept development teams. The differences between the observed 
teams were both minimized and maximized, within the constraints imposed by 
company access. The full range of behavior observed in this study is accounted 
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for in this inductive system diagram. However, the actual implementation of the 
original research design does not preclude the elimination of some plausible rival 
hypotheses which will be discussed below. 

Senior Management Support 

Numerous authors describe the important role senior managers play in 
creating the Market Oriented organization (Shapiro 1988, Kohli & Jaworski 1990, 
Narver & Slater 1990). Gupta & Wilemon (1986, 1988, 1990) specifically identify 
senior management support as a necessary ingredient for creating interfunctional 
integration and cooperation. In this study, it might be argued that those teams 
using Concept Engineering received, or at least could be perceived as receiving, a 
higher level of support from their senior managers than the teams which did not 
use Concept Engineering. The original research design attempted to address this 
threat by using pairs of teams from the same division each of which received a 
beneficial treatment. Unfortunately, the actual design implementation precludes 
elimination of this threat. 

Decision Support Process 

White, Dittrich and Lang (1980) found that the more structured the 
interaction during group decision making, the greater the commitment to the 
group derived solution, as measured by the number of implementation attempts. 
Concept Engineering, as a complete decision support process (see chapter 1), is a 
structured decision making process. A plausible rival hypothesis to the TIME to 
MARKET d)mamics presented above relates to the quality of the decision process 
employed by the MARKET oriented teams. 
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Janis (1985; p.l67), based on studies of errors in strategic decision making, 
outlines seven major decision process criteria which are influence the quality of 
individual or group decisions. High quality group decisions; 

1) thoroughly canvass a wide range of alternatives; 

2) take account of the full range of objectives to be fulfilled; 

3) carefully weigh whatever is found out about negative and positive 
consequences that flow from each alternative; 

4) intensively search for new information relevant for alternative evaluation; 

5) conscientiously take account of any new information, even when the 
information does not support the course of action they initially prefer; 

6) re-examine the positive and negative consequences of all known alternatives 
before making a final choice; and 

7) make detailed provisions for implementing the chosen policy, with special 
attention to contingency plans. 

Janis (1985) states it is plausible to assume that failure to meet these criteria are 
symptoms of defective decision making that increase the chances of undesirable 
outcomes. Further, he states that the vigilance pattern of response is more likely 
than others to lead to decisions that meet the main criteria for sound decision 
making. The vigilant decision maker "searches painstakingly for relevant 
information, assimilates information in an unbiased manner, and appraises 
alternatives carefully before making a choice" (p. 184). From this perspective, 
vigilant decision makers, who succeeded in satisfying the above criteria for each 
stage (requirement identification, idea development and concept selection) of the 
concept decision process, will have more successful outcomes. 

The vigilant decision making argument would indicate that 
comprehensive analysis is a sufficient factor for success in the product concept 
decision process. Concept Engineering, as a complete decision support process 
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(see Chapter 1) satisfies Janis' requirements for high quality decisions as 
indicated in the table below. 



Decision Criteria 


Concept Engineering 


1) Thoroughly canvasses a 
wide range of policy 
alternatives. 


In Transforming voices into Requirements (Step 
4) 200-300 requirement statements are usually 
developed. In Concept Generation (Stage 4) 
teams can generate over 300 solution ideas. 


2) Takes account of the full 
range of objectives to be 
fulfilled. 


In Concept Screening (Stage 5) all concepts are 
screened against the set of key customer 
requirements. 


3) Carefully weighs whatever 
is foimd out about negative 
and positive consequences 
that flow from each 
alternative. 


In Concept Selection (Step 14) final concepts are 
screened not only against customer 
requirements (positive consequences) but also 
against organizational, environmental and 
technological constraints (negative 
consequences). 


4) Intensively searches for 
new information relevant for 
alternative evaluation. 


In Collecting the Voice of the Customer (Step 2) 
and Developing and Administering 
Questionnaires (Step 7) active information 
discovery and collection occurs. 


5) Conscientiously takes 
account of any new 
information, even when the 
information does not support 
the course of action they 
initially prefer. 


Transforming Voices into Requirements (Step 4) 
requires every customer statement be developed 
into a customer requirement for possible 
inclusion and development in the product 
concept. 


6) Re-examine the positive 
and negative consequences 
of all known alternatives 
before making a final choice. 


Solution Screening (Step 13) and Concept 
Selection (Step 14) require all developed 
concepts to be evaluated in a matrix against 
customer / environmental requirements. 


7) Make detailed provisions 
for implementing the chosen 
policy, with special attention 
to contingency plans. 


This activity is not formally part of Concept 
Engineering. However, detailed design 
specification activities usually occur subsequent 
to concept approval. 



In this study, those teams which were successful in developing product 
concepts achieving a high degree of commitment from development team 
members and managers were also those teams which completed a 
comprehensive analysis (Concept Engineering) which satisfied the requirements 
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outlined by Janis. However, one team which used Concept Engineering was not 
successful in developing concept commitment. This team had a relative 
emphasis on TIME versus MARKET and was under considerable pressure for 
progress. Although a relatively complete investigation was conducted not all 
participants were active in the entire concept decision process, e.g. three 
members did not conduct customer interviews, two members did not participate 
in idea generation. As a result, a common appreciation of the design objectives 
was not obtained and commitment to the product concept was low. This would 
indicate that the decision process criteria outlined by Janis may be necessary but 
are not sufficient for success in the product concept development process. 

Market Orientation Contingency 

When is too much of a good thing not a good thing? Several authors 
specify contingencies with respect to the effectiveness of market orientation 
(Houston 1986, Gupta & Wilemon 1986, Souder 1988, Kohli & Jaworski 1990, 
Moenaert & Souder 1990, Narver & Slater 1990). Souder (1988) created a matrix 
with customer sophistication on one axis and R&D sophistication on the other 
axis and prescribes different tactics for the twelve resulting cells. Kohli and 
Jaworski (1990) propose that when the general economy is weak there is a 
stronger relationship between market orientation and business performance. 
Gupta and Wilemon (1986) propose that if a firm values being first in the market 
with new products it will benefit from higher integration. Kohli and Jaworski 
(1990) and Moenaert and Souder (1990) propose that market orientation is less 
valuable in environments with technological uncertainty but more valuable in 
environments with consumer or competitor uncertainty. Several authors address 
the issue that the cost of obtaining more information at some point exceeds the 
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value of the benefit to be derived from knowledge gained. (Houston 1986, 
Moenaert & Souder 1990, Narver & Slater 1990). 

Most of the above contingencies address when more market orientation is 
beneficial. However, in the case where technological uncertainty reduces the 
value for market orientation, the basic dynamics outlined in the high level 
inductive system diagram still appear to provide valuable insight, i.e. pressure 
for progress will reduce systematic analysis reducing evidence, credibility and 
substantive accomplishments. 

Conclusion 

This investigation shows that a relative emphasis on TIME creates an 
environment where pressure for progress encourages development teams to 
conduct incomplete analysis oriented to self-interested outcomes during the 
concept development decision process. The resulting lack of design objective 
appreciation and commitment by the development team leads to waste and 
rework in subsequent product development activities. On the other hand, a 
relative emphasis on MARKET can increase design objective appreciation, 
credibility and commitment but increases the time required in concept 
development. These dynamics indicate that increased time spent systematically 
developing a product concept, which remains stable over the balance of the 
development process, results in getting a product to market faster. Schmenner 
(1988) reminds us of the applicability of the fable of the tortoise and the hare to 
product development acceleration. The tortoise won the race with a diligent, 
focused effort and the hare, while very fast, had a pattern of stops and starts in 
his detour-filled route to losing the race. 
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Chapter 5: Time-to-MARKET Management 
Diagnosis 

The dynamics of TIME versus MARKET orientation activities has 
important implications for the management of product concept development 
activities. The emphasis on TIME clearly implies schedule related measurement 
and monitoring. The measurement and monitoring requirements associated 
with an emphasis on MARKET in the expression Time to Market is not so clear. 
This chapter will explore methods^ managers can use to diagnose concept 
development activities in a time-to-MARKET oriented environment. 

Product Concept Decision Process 

The product concept process can be described to follow the general 
decision process structure outlined by Mintzberg and colleagues (1976) in their 
study of unstructured decision processes. In the context of concept development, 
the three general phases of the decision process are: Requirement Identification, 
Idea Development and Concept Selection. All three phases — identification, 
development and selection — were observed, to a greater or lesser extent, in each 
development team in this study. 

In the identification stage, both recognition and diagnosis routines were 
used to clarify and define the requirements which would drive the product 
concept. In the development phase, search and design routines were employed 
to generate ideas which constitute potential solutions to requirements. Finally, in 
the selection stage, screening, evaluation and authorization routines were 
employed either by the development team, or a management team, to select and 
authorize a product concept for additional investment. Janis (1985) indicates that 

^ Metrics presented in this chapter are derived from work conducted with the CQM Research 
Subcommittee on Product Development Metrics (CQM 1992). 
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the effectiveness with which these individual activities are carried out influences 
the outcome of the decision process — the product concept which determines 
75% of the product's lifecycle costs (NRC 1991). 

Janis (1985) outlines seven major decision process criteria which influence 
the quality of individual or group decisions. The table below identifies where in 
the product concept decision process these criterion are most relevant. 





Requirement 

Identification 

- recognition 

- diagnosis 


Idea 

Development 

- search 

- design 


Concept Selection 

- screening 

- evaluation 
-authorization 


1. Canvasses a wide range of 
alternatives 


X 


X 




2. Takes account of full range 
of objectives 




X 


X 


3. Carefully weights 
pros/cons of alternatives 






X 


4. Intensively searches for new 
relevant information 


X 


X 




5. Conscientiously takes 
account of new information 


X 






6. Re-examines pros /cons 
prior to making choice 






X 


7. Make detailed 
implementation plans 









To identify a comprehensive set of customer requirements an intensive search of 
market segment information is required and this new information must be 
incorporated into concept deliberations. To develop an extensive set of potential 
product concept ideas, deliberate exploration must occur around the set of key 
customer requirements. Finally, to determine the best product concept requires a 
systematic comparison of each candidate concept's ability to satisfy key customer 
and organizational requirements. 
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This mapping of decision effectiveness criteria onto the product concept 
decision process identifies high leverage management diagnosis opportunities. 
Specifically, within each phase (identification, development, selection) of the 
product concept decision process, the relevant decision criterion are aligned with 
variables associated with time-to-MARKET orientation. These variables are then 
operationalized in a manner which is practical for use in diagnosing actual 
product concept development activities. 

Requirement Identification 

Mintzberg et al. (1976) observed that the identification phase consists of 
both recognition and diagnosis activities. They defined diagnosis as "the tapping 
of existing channels and the opening of new ones to clarify and define the issues" 
(1976; p.254). Three decision process criteria are proposed to have potential 
relevance in requirement identification activities; the search for new information, 
the range of alternative generation, and the conscientious consideration of new 
information. Based on the research associated with this study. Customer 
Orientation, Crossfunctional Collaboration, and Credibility are proposed as the 
variables with the most direct impact on the relevant decision process criteria. 





Intensive 
search for new 
information 


Range of 
alternative idea 
generation 


Conscientious 
consideration 
of new info. 


Customer Orientation 


X 






Crossfimctional Collaboration 




X 




Credibility 






X 



Customer Orientation represents a willingness to search beyond 
traditional organizational perspectives. Customer Orientation results in 
developing an understanding not only of the stated customer requirements but 
also of the factors which influence those requirements (Kohli & Jaworski 1990). 
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Customer Orientation leads to a more thorough investigation of the customer's 
use environment. 

In Crossfunctional Collaboration, different functional groups work 
together to form a synthesis of their respective p>erspectives. This joint analysis 
of the opportunities leverages the strengths of each functional group in the 
creation of new insight (Shapiro 1988, Gupta & Wilemon 1986). Crossfunctional 
Collaboration, compared to Partisanship, leads to the development of a wider 
range of customer requirements. 

Credibility is a function of information credibility and source credibility 
(Gupta & Wilemon 1988). In this study, it was observed that Credibility was a 
function of Contextual Awareness and Process Participation. Contextual 
Awareness caused development team members to develop empathy for the 
customer which increased the (information) credibility of customer requirements. 
Similarly, Process Participation allowed team members to personally participate 
in the creation of the new information increasing its (source) credibility and 
subsequent utilization. Credibility is required for conscientious consideration of 
new information. 

The Customer Visitation Matrix and a Process Participation Matrix cam be 
used by managers to assess the opportunities for developing customer 
orientation, crossfunctional collaboration, process participation and contextual 
awareness. 

Customer Visitation Matrix 

The Customer Visitation Matrix (Burchill et al. 1992) can assess the degree 
to which the development team is exploring potential market and the 
opportunities individual team members have to develop contextual awareness. 
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Across the top of the matrix, list important customer types,^ e.g.. Lead Users, 
Demanding, Former, Satisfied, etc. Dovsm the side of the matrix list important 
market segments and the spjecific companies to be visited during market 
exploration. In the intersecting cells, write down the name(s) of the development 
team members who conducted the customer visit. In the hypothetical example 
below, the development effort focused on only New England and Mid-Atlantic 
market segments. Additionally, Smith participated in all customer visits while 
Green only visited retailers and not end-users. Ideally, to develop contextual 
awareness participants would visit a representative cross section of customer 
types. 



Customer 
Selection / 
Vistation 


Lead 

Users 


Demanding 


Lost 


Retailers 


Total 


New 

England 


Brown & 
Smith-2 


Smith & 
Jones - 2 


Smith & 
Jones - 1 


Green & 
Smith-2 


7 


Middle 

Atlantic 


brown & 
Smith-2 
Smith & 
Jones-2 


Brown & 
Smith - 2 


Smith & 
Jones-1 


Green & 
Smlth-1 


8 


South 

Eastern 


0 


0 


0 


0 


0 


Total 


6 


4 


2 


3 


15 



Process Participation Matrix 

The Process Participation Matrix can assess the degree to which individual 
members of the development team participated in activities associated with 
concept development. Across the top of the matrix place the departments 
involved in concept development activities. Down the side of the matrix indicate 

^ See Step 1 of the Concept Engineering Manual (Appendix A) for additional description. 
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the stages and steps associated with concept development. In the intersecting 
cells, write down the names and hours of the development team members who 
participated in the various activities. (This process can be automated through the 
use of appropriate job numbers in the labor accounting system.) In the 
hypothetical example below, the total time investment is fairly equal in each 
department. However, the marketing department. Smith in particular, did the 
majority of data collection and requirement generation, while the engineering 
department. White, who did not make customer visits, did most of the 
requirement evaluation. This imbalance in Process Participation and Contextual 
Awareness could adversely impact Requirement Clarity and Credibility. 



Task 


Marketing 


Engineering 


Total 


Data 

Collection 


Person M.H. 

Smith 29 

Jones 11 

40 


Person M.H. 

White 0 

Green 7 

Brown 12 

19 


Dept M,H, 

Mktg. 40 

Eng. 12 

59 


Requirement 

Generation 


Person M.H. 

Smith 42 

Jones Ifi 

52 


Person M.H. 

White 4 

Green 12 

Brown 12 

28 


Dept M.H. 

Mktg. 52 

Eng. 2S 

80 


Requirement 

Evaluation 


Person M.H. 

Smith 10 

Jones 25 

35 


Person M.H. 

White 48 

Green 12 

Brown 12 

72 


Dent M.H, 

Mktg. 35 

Eng. J2 

107 


Total 


127 


119 


246 



Idea Development 

The Development Phase observed by Mintzberg et al. (1976) consists of 
both search and design routines. They indicate four different kinds of search 
behavior: memory, passive, trap and active and two types of design activities 
that result in either custom-made or modified solutions. Three decision process 
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criteria are proposed to have relevance in idea development activities: the search 
for new information, the range of alternative idea generation and accounts for the 
full range of objectives. Based on research associated with this study. Customer 
Orientation, Crossfunctional Collaboration, and Requirement Clarity are 
proposed as the variables with the most impact on the relevant decision criteria. 





Intensive search 
for new 
information 


Range of 
alternative idea 
generation 


Accounts for 
full range of 
objectives 


Customer Orientation 


X 






Crossfunctional Collaboration 




X 




Requirement Clarity 






X 



Customer vs. Stakeholder Orientation represents a willingness to search 
beyond traditional organizational perspectives. Roth well et al. (1974) conclude it is 
desirable "whenever possible" for designers to visit customers to study their 
technical requirements and also the actual operating conditions. Bailetti and Guild 
(1991; p.91-92) cite three reasons for designers to visit customers: 1) added insight 
and useful knowledge, 2) acquiring knowledge depth that facilitates technical 
activities, and 3) increasing designer acceptance of the results. Customer 
Orientation should produce additional information on technical considerations. 

In Crossfunctional Collaboration, different functional groups work together 
to form a synthesis of their respective perspectives. Different functional groups have 
different frames of reference and skills which they bring to focus on aspects of the 
technology-market environment (Van de Ven 1986, Dougherty 1992). These diverse 
perspectives, if integrated, should generate a wider range of alternatives compared 
to those generated from partisan perspectives. Requirement Clarity results from 
understanding the vital few requirements and the relative priorities within this set of 
requirements. Assessing the degree to which design objectives are met first requires 
a clear understanding of the design objectives. Requirement Clarity is a necessary 
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condition for determining if the developed ideas account for the full range of 
objectives. 

The use of the Customer Selection/Visitation Matrix and a Process 
Participation Matrix, reflecting appropriate stages and steps, can provide some 
diagnosis capability of the idea development process. Additionally, the Idea Count 
Chart and the Requirement Utility Matrix can also be used by managers to monitor 
idea generation activities. 



Idea Count Chart 

The Idea Count Chart can assess the range of alternative ideas generated. 
Concept development activities can be decomposed^ in multiple ways, e.g. by 
customer requirement, functional requirement, etc. The Idea Count Chart displays 
the number of ideas generated in each decomposition category. In the example 
below, a customer requirement decomp>osition was used and the number of unique 
ideas associated with each requirement were coimted. In this hypothetical example, 
only one unique idea was generated to address requirement number 5; this may 
represent an area requiring additional idea generation. 



<i> 

.TS 

^ 0 > 
O 0> 

Is 




Customer Requirement Number 



^ See Step 10 of the Concept Engineering nnanual (Appendix A) for additional description. 
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Requirement Utility Matrix 

The Requirement Utility Matrix can be used to assess the relative clarity 
and flexibility of requirement statements. It was observed that the customer’s 
articulation of their need is often ambiguous, does not clearly state the 
requirement and leaves a great deal of flexibility for designer interpretation. On 
the other hand, it was observed that the designer's articulation of a customer's 
need was often a very specific solution; thus greatly restricting designer 
flexibility. Unfortunately, specifying a requirement as a specific solution, while 
very clear, restricts the number of generated ideas to the one solution. To 
develop the matrix, each requirement statement is reviewed and its relative 
clarity and flexibility are assessed and the requirement is plotted in the 
appropriate cell. Requirements in the upper left hand corner of the matrix tend 
to be written as solution statements not requirement statements. Ideally, 
requirement statements have both high clarity and flexibility and would be 
located in the upper right hand of the matrix. 



High 

I 

Example from the Stripping Basket ^ 

C 

CR 1: The basket has a velcro fastener. | 

CR 2: The basket is releasible with one hand .§ 
CR 3: The basket is durable. 

CR 4: The basket is simple. 



7 

6 


CRl 
























CR2 




5 
















4 

3 






























2 






CR4 








CR3 


1 




1 

























1 ^ 2 3 4 5 6 7 



Relative Requirement Flexibility ^ 



:3 

S' 

oc 

> 

'a 

o 

oc 



Low 
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Concept Selection 

The Mintzberg study (1976) identified three routines in the selection phase: 
screen, evaluation-choice, and authorization. Three decision process criteria are 
proposed to have potential relevance in concept selection activities: accounts for 
full range of objectives, carefully weights pros/cons and examination of pros/cons 
prior to choice. Based on research associated with this study. Process 
Participation, Requirement Clarity, and Traceability are proposed as the variables 
with the most direct impact on the relevant decision process criteria. 





Accounts for 
full range of 
objectives 


Carefully 
weights 
pros /cons 


Examines 
pros /cons prior 
to choice 


Process Participation 


X 






Requirement Clarity 




X 




Traceability 






X 



Process Participation represents the active involvement of participants in 
the complete decision process which includes requirement identification and 
idea development, in addition to concept selection. Inclusion in the decision 
process enables participants to understand how their function relates to other 
functions and also their function relates to the overall innovation (Van de Ven 
1986; p.600). Process Participation enables development team members to 
incorporate a more complete range of objectives in their deliberations. 

Requirement Clarity, by definition, includes the key requirements and 
their relative priorities. Without these conditions it would not be possible to 
evaluate pros and cons of concept alternatives. 

Traceability of the decision process and outcomes was important for 
justifying the decision choices. In this study it was observed that the rationale for 
decision choices made early in the concept development process was required 
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during final concept evaluation. Traceability provides the vehicle for ensuring 
these factors are available for consideration during concept selection. 

In addition to the use of the Process Participation Matrix, reflecting 
appropriate stages and steps, the Concept Selection Matrix can be used to 
diagnose final concept selection. 

Concept Selection Matrix 

The Concept Selection Matrix (Burchill et al. 1992) can be used to assess 
relative pros and cons of the generated (selected) concepts against the full range 
of objectives. Across the top of the matrix list the concept alternatives. Down the 
side of the matrix list the design objectives and constraints (organizational, 
technical, environmental, etc.). Select one concept to be the reference datum and 
evaluate the strengths and weaknesses all other concepts relative to the datum. 

In the example below, "1" is much worse than the datum, "3" is the same as the 
datum, and "5" is much better than the datum. In this example, concepts #3 and 
#8 are comparable with respect to satisfying customer requirements but concept 
#8 requires less risk and resources and appears to dominate. 





Reference 

Datum 


Concept 

2 


Concept 

3 


Concept 

4 


Concept 

5 


Concept 

6 


Concept 

7 


Concept 

8 


Concept 

9 


CR#1 


3 


3 


5 


2 


4 


3 


1 


5 


3 


CR#2 


3 


4 


4 


3 


4 


3 


3 


4 


3 


CR#3 


3 


4 


4 


2 


3 


4 


2 


4 


4 


CR#4 


3 


2 


4 


3 


5 


4 


2 


4 


3 


CR#5 


3 


3 


5 


2 


4 


3 


1 


5 


3 


Technology risk 


3 


5 


2 


4 


3 


5 


3 


3 


4 


Competitive risk 


3 


3 


3 


3 


4 


3 


2 


5 


3 


Resource rqmL 


3 


3 


4 


2 


4 


3 


1 


5 


3 


Total 


24 


27 


31 


21 


31 


28 


15 


35 


26 


Average 


3.00 


3.38 


3.88 


2.63 


3.88 


3.50 


1.88 


4.38 


3.25 


Rank 


7 


s 


2_ 


8 


2_ 





2_ 


] 


6_ 
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Conclusion 

The emphasis on TIME in the expression Time to Market clearly implies 
schedule related measurement and monitoring. However, the measurement and 
monitoring requirements associated with an emphasis on MARKET in the 
expression Time to Market is not so clear. In this chapter, I identify high 
leverage opportunities for management attention in the concept development 
process and propose relevant diagnostic devices. These measurement and 
monitoring devices are designed to assist management of the product concept 
decision process in time-to-MARKET oriented environments. 
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Chapter 6: Next Steps and Concluding Comments 



This thesis presents Concept Engineering as a complete decision support 
process for enhancing product concept development, explores the dynamics of 
TIME vs. MARKET orientation in product concept development, and introduces 
Inductive System Diagrams as a variable development and integration 
enhancement for substantive theory generation. In this chapter, I will outline 
some potential next steps for continuing research in each of these areas. 

Concept Engineering 

Concept Engineering is currently being applied to approximately two 
dozen concept development efforts in ten organizations within the Center for 
Quality Management. The expanded application base includes small 
entrepreneurial concerns developing follow-on products, a Fortune 50 company 
on a large scale development effort, and a state government agency in the 
development of a new law. This application base provides a rich comparative 
setting to explore two broad investigative themes — Concept Engineering 
application contingencies and Concept Engineering process improvements. 

Concept Engineering Process Improvements 

There are two general process improvement themes related to Concept 
Engineering which can be explored. The first is improvement to the overall 
process, specifically focused on reducing the decision time. The second area of 
opportunity involves investigating enhancements to particular decision aids, 
specifically the requirement transformation process and Kano's analysis. 
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Concept Engineering Acceleration 

The actual time to complete the Concept Engineering efforts in this study 
was substantially longer than the projected time. Applying the framework of 
Millson et al. (1992), it should be possible to identify potential areas for 
simplification, step and delay elimination, and parallel processing. The 
traditional roadblock for applying these acceleration strategies is the requirement 
for a thorough understanding of the targeted process. Fortimately, due to the 
combined experience of the User's Group, a fairly high level of Concept 
Engineering process knowledge exists. Additionally, the spirit of mutual 
learning, which is a cornerstone of the CQM, enables faster turns of the Plan-Do- 
Check- Act cycle through sharing of experiences across organizations. Therefore, 
within this environment, research to accelerate the development of market 
orientation within development teams would be desirable and possible. 

Concept Engineering Decision Aid Improvements 

A consistent observation during this study was the tendency for 
"requirement" statements to represent specific solutions rather than be a 
reflection of customer needs. The benefit of writing a requirement statement as a 
specific solution is reduced ambiguity. Unfortunately, the expression of a 
requirement as a functional solution severely restricts the designer's flexibility to 
make tradeoffs. It should be possible, using Likert-type scales for example, to 
assess the relative clarity and flexibility of requirement statements. These 
assessment could be made either by the team members themselves or by expert 
judges, either within or outside a organization. These data can be analyzed to 
assess their impact on development efficiency, effectiveness and innovation. The 
results could conceivably have a significant effect on product definition practices. 
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Kano’s analysis (Kano 1984) helps us understand the relationship between 
the fulfillment (or non-fulfillment) of a requirement and the satisfaction (or 
dissatisfaction) experienced by the customer. Extrapolating Herzberg’s (1966) 
motivator-hygiene theory to product quality characteristics Kano discovered that 
the relationship between fulfillment of a need and the satisfaction or 
dissatisfaction experienced is not necessarily linear but can be separated into four 
main categories: attractive, one-dimensional, must-be and indifferent. For 
example, when an attractive item goes unfulfilled the result is not dissatisfaction 
but the absence of satisfaction; in contrast, fulfilling a must-be item does not 
produce satisfaction. Currently, it has been observed that the construction of the 
questions and response scales associated with Kano's analysis is problematic for 
some development professionals. Additionally, Kano's analysis has not been 
systematically evaluated (or at least published in English language academic 
journals) in the context of traditional marketing research techniques. This is an 
area of investigation with potentially significant implications for product 
development resource allocation decisions. Preliminary work is underway for a 
collaborative effort between myself and Duncan Simester (1993 MIT-Marketing 
Ph.D. candidate) to conduct this more systematic analysis. 

Concept Engineering Application Contingencies 

With approximately two dozen Concept Engineering applications 
underway, covering the spectrum of market-technology uncertainty, a more 
thorough understanding of the contingencies associated with applying Concept 
Engineering can be developed. Current applications range from simple product 
line extensions, to radically new product concepts, to the development of new 
state laws. This environment can provide a rich comparative setting for the 
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development of contingency theories related to the product concept decision 
process. 

All of the completed Concept Engineering applications have been product 
extensions for existing markets or existing technologies. However, some 
members of the User's Group believe Concept Engineering is best suited for 
developing products for new markets or technologies. One effort currently 
underway is pursuing a completely new product concept which they hope will 
create entirely new markets. Another effort is exploring px)tential market 
applications for new technology. These extremes, anchored by base cases in 
market- technology extensions leads to a potentially productive line of 
comparative research addressing the contingencies of Concept Engineering in 
particular and Market Orientation in general. 

The effort to develop a new state law represents a service application in 
which constituencies have clearly conflicting positions. Prior Concept 
Engineering efforts have investigated diverse market segments, i.e. North 
America, Europe and Asia. However, it was assumed in those efforts, that if a 
clear conflict developed, multiple products would be developed or the decision 
choice would be made on the basis of the largest profit potential. In the case of 
the state law, it is not possible to develop multiple versions and it is not obvious 
how "profit potential" would be assessed. However, based on the work of the 
Harvard Negotiation Project (Fisher & Ury 1981), a dominant strategy for 
successful negotiations is to focus on the common interests instead of conflicting 
positions. In the state case, it has been possible to identify and articulate the 
common requirements of all constituencies in addition to their requirement 
differences. This case represents a potentially useful process application of 
Concept Engineering which has not been previously explored. 
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Time-to-Market Dynamics 

Chapter Four of this thesis presented numerous propositions related to 
Time-to-Market dynamics which need to be validated. Some of the constructs 
identified as influential have been operationalized and investigated in other 
studies (). Other constructs (e.g. traceability, process participation, contextual 
awareness) have not been investigated and their significance on product concept 
development has not been statistically validated. Chapter 5 attempts to 
operationalize some of these variables from a managerial perspective which may 
be insufficient for a study attempting to apply traditional academic standards of 
statistical significance. Additionally, even after the hurdle of developing 
multiple construct operationalizations is overcome, the data collection process 
will be formidable. 

My experience, at every company in this study, indicates that product 
concept development data is not systematically collected, if it is collected at all. 
Concept development activities are traditionally ad hoc and corporate product 
development data collection systems typically begin after concept approval not 
before. However, sufficient data may be available to assess the basic proposition 
that credible design objectives result in stable product concepts and the resulting 
reduction in misdirected effort reduces total development time. 

Many companies use a product development phase review process with a 
formal "Concept Approval" stage. Conceivably at this point, a product definition 
exists which can be assessed for clarity and credibility, e.g. using Likert-type 
scales. Concept stability can be determined by reviewing engineering change 
notices assuming the records exist and it is possible to distinguish those 
engineering changes related to concept instability from those related to other 
sources, i.e. manufacturing requirements. This information can be compared to 
actual development time versus the original development schedule. This final 
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comparison is also dependent upon several assumptions regarding schedule 
forecasting reliability and project characteristics. However, a test for collecting 
the data described above, with the assistance of the company's Product 
Development Quality Assurance Department, was conducted in one company 
from this study. This limited data collection effort was difficult but does indicate 
the feasibility of investigating some of the Time-to-Market propositions utilizing 
empirical data. 

Another path to model validation could be through system dynamic 
simulation. In the system dynamic approach, model validation follows from a 
multi-method analysis of computer simulations (Forrester & Senge 1980, Maas & 
Senge 1980, Richardson & Pugh 1981, Sterman 1984, Barlas 1989, Barlas & 
Carpenter 1990). System dynamicists have a long history of successful 
simulation analysis of development project management (see for example: 
Roberts 1964, Richardson & Pugh 1981, Abdel-Hamid & Madnick 1991, Sterman 
1992). This previous work, assesses the impact of project changes on 
downstream development activities given initial tasks. However, as the existing 
work does not model the early concept development activities addressed in this 
research an opportunity exists for this research to extend previous dynamic 
models of project management. 

The Time vs. Market inductive system diagram would serve as the 
conceptual model around which a formal computerized model can be built. 
Model formulation is the process of transforming the conceptual model into 
equations which increase the precision with which the system structure is 
specified. This precision, which is a necessary condition for conducting the 
computer simulation and analysis, also reduces the ambiguity of the causal-loop 
diagram (Richardson and Pugh 1981). For example, in this research it is 
proposed that reductions in Development Time would reduce Pressure for 
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Progress (Chapter 4, Proposition 18). Formulating this relationship might 
produce the equations below: 

Pressure for Progress = 

1 + (Expected Development Time - Scheduled Development Time) / Scheduled Development Time 
Expected Development Time = Perceived Work Remaining / Perceived Work Rate 
Perceived Work Remaining = Scheduled Work - Work Completed + Recognized Rework 
Perceived Work Rate = (Work Completed - Recognized Rework) / Labor-hours Invested 

These four equations clearly demonstrate how the formulation process 
fleshes out the skeletal structure of the inductive system diagram by specifying 
the system substructure of levels and rates. Formalizing the detailed dynamics 
of the entire Time vs. Market inductive system diagram will allow for a more 
precise definition of the relationships and subsequent simulation analysis of the 
propositions presented in this research. 

Inductive System Diagrams 

Two research themes could be pursued directly from the initial work on 
Inductive System Diagrams. First, a more extensive reliability assessment can be 
conducted and second, research related to enhancing the power of the diagrams 
should be pursued. 

The reliability assessment of Inductive System Diagrams needs to be 
conducted on a larger sample of testers, some of whom are experienced 
qualitative researchers. Additionally, the assessment of the diagrams should be 
conducted by a panel of trained evaluators rather than a single person to increase 
result reliability. Finally, given larger sample sizes statistical analysis of the 
results can be conducted. 

The rate-to-level limitations of causal-loop diagrams has been addressed 
by several systems dynamists through the use of flow diagrams (Forrester 1971, 
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Goodman 1974, Richardson 1981) and Policy Structure diagrams (Morecroft 
1982). Prior attempts at representing this additional structural detail 
unfortunately make the schematic much more difficult to comprehend by the 
uninitiated. However, it should be possible for the analyst to employ these 
structural insights in the development and description of their models even if the 
detail is absent from the presentation schematics. 

Additionally, Inductive System Diagrams can be extended by 
incorporating reference mode analysis into the development process. Reference 
modes clearly specify the dynamic behavior of interest in the system under 
investigation (Randers 1980, Richardson and Pugh 1981). Usually reference 
modes are based on actual historical data but they can also be created from 
expert assessments (Randers 1980, Richardson and Pugh 1981). Reference modes 
can be described either graphically or verbally but they must indicate the 
appropriate time dimensions of the variables described (Randers 1980, 

Richardson and Pugh 1981). Reference modes can help identify which variables 
should appear in the model (Randers 1980). Therefore, reference mode analysis 
could assist not only in the development of variables through theoretical 
sampling and coding but also in the elimination of variables during diagram 
integration. 

Concluding Comments 

At the highest level of abstraction, the basic lessons learned from the analysis of 
the Time-to-Market dynamics we were taught long ago: "Do it right the first time; 
because if you don’t make time to do it right, you’ll have to make time to do it over." 
One major contributing factor to the dysfunctional, imintended consequence of 
product concept development acceleration is the lack of product concept decision 
process understanding. Concept Engineering, as a complete decision support process. 
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clearly defines the steps and transitions associated with the product concept decision 
process and therefore has the potential for accelerating the development of market 
orientation within product development teams. 

The development of Concept Engineering, the Time-to-Market analysis and 
Inductive System Diagrams all result from "cross-functional" collaboration. Without a 
common commitment to mutual learning between the initial companies and 
researchers at MIT, Concept Engineering would not have evolved into a complete 
decision support process. Without the companies granting the researcher extensive 
access to their product development activities, participants and managers, the rich 
comparative setting, essential for generating substantive grounded theory, would not 
have developed. Without the inter-disciplinary involvement of Operations 
Management faculty and Behavioral Studies faculty, the dialogue which resulted in 
Inductive System Diagrams would not have developed. Finally, without an overall 
commitment, by the thesis committee, to a partnership between industry and academia 
in the research of Total Quality Management, this thesis would not have developed. 
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Preface 



This manual is designed to assist organizations in focusing on their 
customers' requirements in developing design concepts for products or 
services. Concept Engineering is a process for determining the customer’s 
key requirements, creating a measurement strategy for assessing 
compliance with the requirements, and developing a strong product 
concept that satisfies the requirements. 



Document History 

The Concept Engineering method and manual have been developed in 
the true spirit of mutual learning and collaboration between member 
companies of the Center for Quality Management (CQM) and MIT. The 
material presented in this manual is the result of many Plan-Do- 
Check- Act improvement cycles applied to both the teaching and use of 
this method. 

• The foundation for Concept Engineering began with a series of 
lectures by Professor Shoji Shiba at MIT and the CQM in the fall of 
1990. 

• The first outline of the process was developed by Gary Burchill 
(USN/MIT) in the winter of 1990. 

• The first outline of the manual was developed by Gary Burchill 
(USN/MIT), Diane Shen (BBN), Ron Santella (GenRad), and Rich 
Lynch (Analog Devices) in the spring of 1991. 

• The first version of the manual (CQM 7P),written by Gary Burchill 
(USN/MIT), Diane Shen (BBN), and Ron Santella (GenRad), was 
published in November 1991. 

• The second version of the manual (CQM 71), written by Gary 
Burchill (USN/MIT), Diane Shen (BBN), Erik Anderson (Bose), 
David Boger (Bose), Chris Bolster (MIT), and Bill Fetterman 
(Analog Devices), with editing assistance from Kenny Likis (BBN) 
and Deborah Melone (BBN) was published in September 1992. 

• The authors would like to thank the development teams in BBN, 
Bose, Analog Devices, and Polaroid for their feedback on Concept 
Engineering and Dave Walden (BBN) for his encouragement and 
critique. 
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Critical Assumptions 

Two critical assumptions have been made in the preparation of this 
manual: first, that the users of this manual are familiar with and 
comj::)etent in the KJ method; second, that the scope (minor product line 
extension, major new product introduction, etc.) of the development 
project has been at least preliminarily defined before entering step one. 
Exploration Planning. 

If the first assumption does not hold, we recommend that KJ skills be 
obtained (through CQM courses) to maximize the potential benefit of 
this method. If the second assumption does not hold, i.e., the scope of 
the development project is very broad or vague, several iterations of 
the early steps of this method may be required before a market 
opportunity is identified. 



The example used throughout this 
manual 



This manual is designed to provide users with a quick introduction to 
the underlying concepts and methodology for each step. Steps are 
supported with examples, tips, and worksheets where appropriate. 

The appendix includes a glossary and further detail about selected 
concepts. 

All examples come from a case study provided by Gary Burchill from a 
project on which he worked at MIT. The product developed by the MIT 
team was a saltwater flyfishing stripping basket. 

A stripping basket is a device used by saltwater fly fishermen to collect 
their line before they cast out the line. Typically it is a store-bought or 
home-constructed plastic container with four sides and a bottom, which 
is strapped to the waist or chest of the fisherman. Before casting, the 
fisherman lays the fishing line into the container so the line will play 
out easily when the cast is made. The process of retrieving the line and 
placing the line in the container is called "stripping." 

The goal of Gary Burchill and his MIT colleagues was to design a better 
stripping basket. Their basket was praised by the Outdoors Editor of 
the New York Times in his feature on Sunday, September 1, 1991. First 
year sales of their product has been ten times that of the product it 
replaced. 

Other examples of the application of Concept Engineering are 
available from various CQM companies. 
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Introduction 



Why Bother? 

A recent study conducted by the National Research Council states that 
over 50% of a product’s life-cycle costs are determined in the concept 
formulation phase of product development, and that approximately 
75% of life-cycle costs are comnnitted by the end of concept validation. 
Yet many companies spend little effort in these phases of product 
development and do not have effective n>ethods for developing product 
concepts which they are confident will satisfy their customers. Unless 
people have an effective method for understanding the customers' needs 
and finding a product concept to meet them, companies will be locked 
into unsatisfactory concepts which drive the life-cycle costs of their 
products. Concept Engineering is a process designed to provide such a 
method. 



Life Cycle Cost Commitment 
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The National Research Council study outlines the troubled state of US 
engineering design skills. Interviews with senior managers associated 
with product development at CQM member companies reinforce the 
findings of the National Research Council report. These senior 
managers were asked several questions, one of which was to describe 
images that come to mind when they think about their product 
development process. These observations, captured by the KJ shown on 
the following page, emphasize the importance of establishing a 
concept engineering process. 



Market-In Model 



Concept Engineering is built on two models introduced to the CQM by 
Professor Shiba: Market-In and WV problem solving. 

The ”Market-In” attitude expands the horizon of how people and 
organizations view their job resp>onsibilities. The output of one’s effort 
is not the end objective of work, but instead the means by which to 
satisfy the customer. The end objective is customer satisfaction. 



”Market-In” 




In contrast, the product-out attitude defines its task as first designing 
and building the product or service, and then convincing customers that 
the product or service really meets their needs. 
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What are the images of product development 
activities within interviewed companies? 
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WV Model 



The WV model,Professor Shiba's extension of Professor Kawakita's W 
model, is usefulfor framing the product development process. It starts 
with a broad-based exploration of essentials and, by alternating 
between the level of thought (ideas and concepts) and experience 
(data), key issues are progressively defined in ever finer detail. 



7. Reflect on Prcx^T| | 6. Standardize | 




Concept Engineering 

Concept Engineering is a customer-centered process for clarifying the 
"fuzzy front end” of the product development processthat comes before 
detailed design and implementation. It is a conceptual model, with 
supporting methodology, for developing product concepts. The process 
alternates between the level of thought and level of experience in a 
way that allows participants to understand what is important to 
customers, why it is important, and how it will be measured and 
addressed. It is a customer-centered process of data collection and 
reflection designed to develop product concepts that will meet and 
exceed customer expectations. 
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Concept Engineering Stages 
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stage 1 : Understanding the Customer's 
Environment 



In stage 1, an exploration plan is developed based on the project scope, 
which identifies the customers to be visited and the information, 
broadly defined, which is being sought. The customer visits are 
conducted with an emphasis on collecting notes on verbatim customer 
statements and field observations. Then, the development team 
develops a mental model of the customer’s environment to create a 
contextual anchor for downstream development. 



stage 2: Converting Understanding into 
Customer Requirements 

In Stage 2, the customer visit notes are analyzed to uncover the customer 
requirements (both explicit and latent) and the requirements are 
transformed from the language of the customer into the language of the 
company; from affective language into concrete statements. The vital 
few requirements are selected from the useful many and arranged in 
various combinations to create new insight. 



stage 3: Operationally Defining Requirements 

In stage 3, characteristics of the vital few requirements are 
investigated with customers. Additionally, metrics are developed 
which will be used to measure quantitatively how well the 
requirements are met. Finally, all of the information and insight 
which has been developed is clearly and concisely displayed in one 
document. 



stage 4: Generating Concepts 

In Stage 4, the complex design problem is decomposed into smaller, 
independent subproblems. An exhaustive list of solutions (both feasible 
and infeasible) is created for each subproblem. Subproblem solutions 
are then combined to create solution concepts. 

stage 5: Selecting Concepts 

In stage 5, the most promising solution concepts from Stage 4 are 
compared, in a structured process, against the customer requirements. 
The concepts which best fit the customer’s requirements and the 
company’s development capabilities are then selected for 
implementation. 
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stage 1 : 

Understanding 
the Customer's 
Environment 



The first of the five Concept Engineering Stages is Developing an 
Understanding of the Customer's Environment, Critical to this stage is 
planning and scheduling of all downstream Concept Engineering 
activities. This stage consists of developing a plan for your team's 
exploration, doing the exploration, and using your team's interviews to 
develop a contextual anchor from the images of the customers' 
environment. This contextual anchor is actually a KJ diagram of images 
of the customer's environment. The Image KJ is a link to the customer's 
real world and acts as a way to ground all future decisions in the 
customer's environment. 

As shown in the following figure, there are three specific steps included 
in Stage 1: Step 1, Plan for Exploration; Step 2, Collect the Voice of the 
Customer; and Step 3, Develop a Common Image of the Customer's 
Environment. 
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Concept Engineering Stages 



Stage 1 Steps 




The five Stages of Concept Engineering are shown above on the left and 
will be repeated throughout this document as a way to keep track of 
where each stage is in relation to the entire process. 
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step 1: Plan for Exploration 

The purpose of Step 1 is for the team to discuss and understand the scope 
of the project and to develop a road map of future project activities. In 
this initial planning, your team agrees on the scope of the exploration 
theme, selects an exploration method, plans for and schedules the 
entire Concept Engineering process, and specifically plans for the data 
collection process. Planning for exploration will aid your team 
throughout Concept Engineering by supplying direction and purpose. 



Understanding the Scope of the Project 

Here the team must make a series of decisions defining the breadth of 
your exploration, starting with an exploration theme. 

The exploration theme is an important device for setting project scope. 
For example, the following potential exploration themes would result 
in projects of different levels of scope and complexity. 

A. "What are the most impxjrtant customer requirements for 
flyfishing? 

Or: 

B . "What are the most imp>ortant customer requirements for salt-water 
flyfishing?" 

Or: 

C. "What are the most important customer requirements for casting in 
a salt-water environment?" 

Or: 

D. "What are the most important customer requirements for a 
stripping basket?" 

Example A defines the scope as the entire sport of flyfishing. Example 
B narrows the scop>e to salt-water flyfishing. C narrows it even further 
to casting in a salt-water environment, and D is focused specifically on 
requirements for a stripping basket. 

In example B, note how the focus of the team must change if only a few 
words of the theme are changed. If the word "salt-water" is added, 
the focus changes significantly. 

The team should avoid limiting itself only to traditionally defined 
processes or services. Compare examples C and D above. Example D 
leads the team into the "solution space" of a stripping basket as an 
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answer to fishing line management problems. Example C, on the other 
hand, stays out of the "solution space" and remains in the "problem 
space" by directing the team to focus on the activity, not a solution to 
one of its problems. This subtle difference can significantly alter how 
the team develops its interview guidelines, thereby impacting what 
the team discovers. Whenever possible, do not limit your team's 
activity by adhering to conventional definitions of processes or 
practices. 



Select Exploration Method 

There are a number of methods for collecting data from your customers: 
in-person interviews, telephone interviews, laboratory observation, 
etc.. The figure below plots a number of different exploration methods 
on a two-dimensional map showing degree of intervention with the user 
vs. proximity to the user environment. 
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Unstructured interviewsjnvolve high intervention with the customer — 
an interviewer asking questions and follow-up questions based on the 
answers. These may occur either close to the customer’s environment, as 
in a visit in the customer’s office, or further from the customer's 
environment as in an interview conducted over the phone. 
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Process Participation involves some intervention and is by its nature 
closer to the customer's environment. Contextual Inquiry is a method 
used by Digital Equipment Corporation which involves sending 
engineers to the field to engage in a dialogue with the customers while 
they observe them using the product. 

Participant Observation involves low intervention and can range from 
close to far from the customer’s environment. Observing customers from 
behind a one-way mirror is an example of this kind of research. 

Different methods are appropriate for different purfx)ses. Keeping in 
mind that one purpose of Concept Engineering customer data collection 
is to extract images, that is, visualizations by customers using the 
product or service in their own environment, you should choose the 
method for exploration that best suits that need as well as other 
realistic constraints. In this document, we will focus mainly on in- 
person interviews at the customer’s site, which include observations of 
their use environment. The major concepts and guidelines discussed, 
however, will be applicable to other methods as well. 



Planning for Concept Engineering 

It is imp)ortant to have a well-designed plan of action for the entire 
Concept Engineering process. Concept Engineering consists of individual 
work and many team working sessions. Meeting coordination can be a 
powerful determinant of the team's momentum and eventual success. 

The team should agree upon a schedule and all members should plan for 
these meetings in advance. We recommend following Professor Shiba's 
advice to schedule a project completion date first and then plan 
backwards, filling in the activities and dates. The schedule for 
Concept Engineering needs one or two weeks of dedicated time after 
data collection in order to process the data. EHiring some steps, weekly 
meetings are needed. 

The Concept Engineering schedule will vary considerably depending 
upon the complexity of the team's project and the difficulty of reaching 
customers and scheduling interviews. However, all team-dependent 
activities should be scheduled into as short a time period as fx)ssible 
after the interviews are completed. The following guideline for task 
planning is a rough estimate of time required for each major task. The 
elapsed time is fifteen weeks. Some teams have completed Concept 
Engineering in fewer weeks; some have taken longer. 
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Gantt Chart of Concept Engineering Activities 
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Planning for Data Collection 

Planning the specifics of data collection requires a thorough 
understanding of the types of customers in the markets that your team 
intends to explore, as well as an understanding of the exploration 
methods your team will use. Deciding the who, what, when, where, 
and how of data collection enables the team to consider how they will 
divide up market segments. 

How Many Interviews? 

Open-ended customer interviews are intended to explore the market 
and learn about customer needs. One of Professor Kawakita's principles 
emphasizes the importance of using a small number of qualitatively 
rich cases. This principle is supported by the work of Professors 
Griffith and Hauser, who hypothesize that 10 to 20 interviews are 
sufficient if conducted with knowledgable customers. Twelve to fifteen 
interviews is a target that is realistic and not overly cumbersome. 
However, if you believe you have distinctly different market 
segments, you may need a minimum of 10 interviews per segment. 

Which Customers to Select? 

Diversity in selecting customers is important in order to explore the 
market broadly. A Customer Selection Matrix, shown below, is a 
helpful tool to search for diversity among the customers you visit. 

Customer Selection Matrix 
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Categories of Customers 

The categories down the left side of the matrix cover typical market 
segments. These may be geographically based, or based on any other 
category of market segmentation that is typical for your field of 
exploration. In the example above, fly-fishing markets can be 
segmented into: New England Salt-Water Flyfishing, Caribbean Flats 
Flyfishing, Salmon Flyfishing, Steelhead Flyfishing, Trout Flyfishing 
and Distributors. 

At the top of the matrix your team thinks about your customers in a 
slightly different way and lists categories of customers, for example, 
Lead-Users, Demanding Customers, Happy Customers, Unhappy 
Customers, Customers We've Had but Lost, and Customers We've Never 
Had. The categories across the top of the matrix will help ensure that 
you are not simply approaching those customers who are easiest to 
approach, but those who are, potentially, most useful to your Concept 
Engineering process. 

Lead Users, a concept developed by Professor Eric Von Hippel, have 
needs which typically foreshadow the general demand of the market. 
Additionally, they orien have some prototyp>e experience; that is, they 
have worked around existing product constraints to solve their problem. 
Therefore, Lead Users are particularly helpful in exploring the 
opportunities, as their prototype experience gives them a potentially 
greater ability to perceive and express needs. 

Using th0 Matrix 

Once the categories are listed, begin filling in customer names in each 
of the cells. Once all cells are filled in, select customers who are most 
appropriate for your project. 

It is not necessary that the team visit and interview a customer from 
each combination of categories; indeed, that would put the team well 
over the suggested 12 to 15 interviews. A general guideline from 
Professor Ed McQuarie, Santa Clara University, is to have three to four 
interviews in cells you consider most important and fewer, if any, 
interviews in the others. 

Who Conducts tho Visits? 

Interviews should be conducted in pairs, preferably by two people from 
different functions in the organization, i.e., one from marketing and one 
from engineering. There are many reasons for using cross-functional 
interview teams. For example, the marketing person may be more 
likely to focus in on the customer's statements of needs; the engineer 
may focus on the customer's solutions to a difficult technical problem. 
Together they have a better chance of gathering a 360-degree view of 
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the customers world. Additionally, as engineers seldom spend time in 
the field, this is an excellent opportunity to put the engineer face to 
face with the customer. This opportunity to visit and explore the 
customer’s environment can build the team's empathy for the customer. 



Exploration Principles 

Market exploration through in-depth, open-ended customer interviews 
is a marketing research method which is different from methods 
intended to validate market hypotheses. Customer interviews can be 
used to develop hypotheses; traditional market research methods are 
still needed to test hypotheses, gather demographic data, etc. As a 
different type of market research, customer interviews follow different 
guidelines than quantitative market research tools. Professor 
Kawakita has developed five guiding principles for collecting 
qualitative information: 



1. Take a 360-degree perspective. Walk around the situation from 
many different angles. Do not have a hypothesis about what you 
will hear; keep an open mind in order to explore broadly, 

2. Have a stepping-stone agenda. Do not schedule customer visits 
back to back. Allow for the jx)ssibility of an unexp>ected visit 
opportunity. Use an interview guide loosely; follow the path the 
customer creates. 

3. "By Chance”. Utilize chances - but you can create chances if you 
are sensitive ; concentrate on the problem or opportunity in order to 
see chances to learn more. Louis Pasteur says, "chance favors only 
the prepared mind." 

4. Use your intuitive capability. Intuition is the tool of discovery, 
logic is the tool for proof. Human intuition has great capability to 
find somethind new. If your intuition is telling you to pursue a topic 
with a customer, follow your intuition. 

5. Qualitative vs. quantitative data. Numbers are not so important; 
cases and personal experiences are important. Diversity of insights 
is more important than the number of times you hear the same 
information. 

In addition to Kawakita's principles, Peter Dnicker's work on sources of 
innovation is a helpful guide. Drucker states that innovation is most 
likely to be found in surprises, incongruities, and process holes. 
Customer interviews can be rich sources for discovering and exploring 
these innovation opportunities with a 360-degree perspective. 
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Developing the Interview Guide 

The Interview Guide is a small set of open-ended questions and sub- 
questions you can use as a guide during the interviews. It is only a guide, 
not a strict questionnaire that must be rigidly followed. It acts as a 
reminder to ensure that you cover all of the important topics. This is 
not to suggest that the guidelines restrict you from taking a different 
path during the interview. Indeed, it must be left to the interviewer to 
determine when there is an opportunity to veer off from the guidelines, 
either for broader exploration or to gather specific, detailed 
information. These opportunities usually occur in later interviews 
when the interviewers’ sensitivity and interview skill is increased. 

The questions which make up the Interview Guide can be generated in a 
number of ways. The Net-Touching process, outlined below, helps 
ensure that appropriate breadth and depth of coverage is developed. 

1. Brainstorm answers to the question, "What do we want to learn on 
our visits?" Write each thought on a 3"x3" label. (This could be 
done as individual work before the group meets). 

2. Prepare the meeting room as you would for a KJ. 

3. Have the team leader or facilitator randomly select a label, read 
it out loud, and post it to the board. They then ask other team 
members to provide labels of similar questions. 

4. When there are no more labels on the topic, write a title for the 
group, in the form of a question, which is one step higher on the 
ladder of abstraction. 

5. The team leader asks for a label on a new topic. 

6. Continue this process until all original labels are posted to the 
board, either in groups or as lone wolves. 

7. Continue grouping (by classification) the question-groups, writing 
titles in the form of questions until a small number (4 to 6) of groups 
are formed. 

Professor Shiba suggests ensuring that the questions cover the following 
categories. For any that are missing, add a new question. 

• Past problems or weaknesses of the product or service — these are 
useful in providing insight into the customer’s perceptions. For 
example: What are the weaknesses or problems you’ve experienced 
with this (or this type oO product or service? 

• Current considerations associated with purchasing or using the 
product or service — these can be helpful in understanding the 
customer’s expectations. For example: What do you think of when 
selecting this product or service? 

• Future enhancements of the product or service — this provides 
additional emphasis and detail on points previously discussed. For 
example: What new features might address your future needs? 
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• Scenes or imag es of the customer's use environment — these are 

essential for grounding all future development actions in the context 
of the customer's experience (Step 3). Some examples: What are 
the images that come to mind when you visualize the use of...? or. 
When you think about using this ... what are the scenes that come 
to your mind, or. If I had a video camera and recorded a scene of you 
using this product (or working in the lab, or struggling with a 
critical problem, etc.), what would 1 see? 

Compile the questions and sub-questions from the net-touching, adding 
any that come from reviewing the four categories of questions, into an 
interview guide. Think about a jx)ssible sequencing of the questions. 

interview Team 

Interviews should be done in pairs: that is, two people interviewing one 
customer. One of you is the questioner, the other takes verbatim notes of 
the customer's responses to questions. Notetaking does two things: it 
shows respect for the customer, and it captures the data. The 
questioner takes notes of observations and areas to explore. In general, 
the questioner stays engaged with the customer and does not take too 
many notes. The notetaker stays focused on writing the actual words of 
the customer (not a summary of the customer's thoughts) and should not 
ask too many questions. These roles of interviewer and notetaker can be 
reversed between interviews, but should not be changed during an 
interview. 

Practicing 

Practice interview sessions have proven to be jx)werful in developing 
interview skills. Role play an interview. Have one person as an 
interviewer, one as a notetaker, and one as a customer. Conduct the 
entire interview using the interview guide. Watch your body language 
to be sure you px>rtray a message of interest throughout the interview. 
Debrief after the interview and discuss what worked well and didn't 
work well. Rotate roles so that each team member gains experience as 
a questioner and as a notetaker. 

Swim in Shaiiow Water 

Do a dry run of the interview process on familiar customers or internal 
people, and revise the guideline as necessary. This is a critical step; do 
not underestimate its impx)rtance. 

Doing Your Homework about Customers 

In order to show respect for the customer, before conducting visits , make 
sure that you are well briefed in the background of the customer, the 
products or services the customer purchases from your company, and 
additional information that may be meaningful to your visit. 
Understanding what your customer’s business is and demonstrating that 
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understanding in the interview process will go a long way toward 
establishing a comfortable rapport for the visit. 



Tips 

• Make sure to include the sales force as appropriate in planning and 
scheduling customer visits. 

• Do as many practice interviews and shallow-water swims as time 
allows. The investment will pay off when you are doing customer 
interviews. The team will spend much time and effort on doing the 
interviews, and the success of the future product depends upon the 
data brought back from the interview. 



Completion Checklist 

At the end of Step 1 you should have: 

• A schedule for the project 

• A Customer Selection Matrix complete with the names of customers 
to be visited 

• An interview guideline which has been tested 

• Team members with training and practice in interviewing skills 

• A list of who will visit whom 
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step 2: Collect the Voice of the 
Customer 



The purpose of this step is to gather the customer data which will 
drive all subsequent Concept Engineering activities. 

A useful concept for this step is Professor Shiba’s ’’Swimming in the 
fishbowl.” It depicts the fishbowl as the user’s (customer’s) 
environment, with the swimmer (interviewer ) entering, swimming 
around, exiting, and reflecting upon what was learned. In contrast, 
traditional market research looks at the market with developed 
hypotheses, essentially looking from the outside and measuring 
behavior in the fishbowl. Customer Visits and Contextual Inquiry are 
methods by which people who will make decisions about the product or 
service jump into the fishbowl, swim around and see what is going on, 
and then jump back out of the fishbowl to analyze what they saw. 



Swimming in the Fishbowl 
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Scheduling Visits 

The interviewing process requires the careful balancing of the 
interview schedule around customers’ availability and your needs. 
Generally the team members will have to adapt their schedule to the 
customers'. 

Prior communication with the customer is important to the success of the 
interview. A letter confinning the date, time, purpose, and the agenda 
should be sent to each customer. (It is also a nice touch to fax the 
interview guide to the customer a few days before the visit.) The degree 
to which you are open about your intention, and the degree to which you 
and your team honor the customer in a number of ways will determine 
the extent to which the customer will be open and honest in return. This 
effort at building rapport can be particularly valuable if you need to 
make additional contact. Honoring the customer at all times should be 
of highest priority. 

Schedule 60 to 90 minutes for each interview, but be flexible. If you 
schedule more than one interview at a site, allow for approximately 2 
hours between interviews. Even if you intend to conduct only one 
interview in a day, block off at least 90 minutes after the interview for 
debriefing with your interview partner. Preferably this will occur 
immediately after the interview concludes; conduct the debriefing in 
the parking lot if necessary. 



Conducting the Visits 

Visiting and interviewing customers is not a task to be taken lightly. 
Remember that every time you interface with a customer in the Concept 
Engineering process, it is for your education, not the education of your 
customer. Customer interviews are not opportunities for sales calls. It's 
your listening skills, not presenting skills, that are going to be tested on 
these visits. 

In fact, the listening required on a customer visit forces you to: 

• Be open and receptive to the customer’s opinions and feelings. 

• Suspend all judgments. 

• Accept whatever the customer says 100%; never argue! 

One of your goals in the interview is to get beyond the first or obvious 
answers to the essential data. Often the customer's first response to a 
question is just the tip of the iceberg. Following up with probing 
questions is necessary to get to the bottom of the iceberg where the 
majority of the data is; explore the customer's thoughts in depth. 
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For example, the customer might give you a solution idea. Below the 
tip of the iceberg is the problem the customer is trying to solve. Explore 
this topic until you understand the nature of the problem in addition to 
the solution the customer has mentioned. The customer's solution 
should be recorded because it may be useful in Generating Solution 
Concepts, but in order to learn about the customer's need, you must 
understand the problem hidden below the solution. 

Finally, remember to thank the customer for the opportunity they 
have given you to learn. 



Debriefing 

As was mentioned earlier, you should schedule 60 to 90 minutes after 

each interview to debrief. You may end up using the time to conduct a 

spontaneous interview. If this happens, you should still debrief as soon 

after the interview as possible. 

Follow these steps immediately after the interview: 

1 . Make a copy of the notes. 

2. Discuss general observations for a few minutes. 

3. Read notes carefully, filling in gaps with the customer's actual 
words if you can recall them. 

4. Discuss the interviewer's questions and follow-up questions. Note 
what worked well and didn't work well. 

5. Discuss and note insights about your customers, their environment, 
their needs, etc. that you gained from this interview. 

6. Think about improvements to the interview guide and note these. 

When you get back to the office: 

7. Share observations with other team members. 

8. Type up the verbatim customer notes, creating a transcript of each 
interview as soon as possible. 



Image Collection 

As discussed earlier, images are the scenes or descriptions of product use 
or the emotion associated with the product’s use that are generated in 
the interview process. They may also include your observations. 

Read through each customer interview transcript and pull out the 
images of the customer’s environment by looking at each customer 
statement and thinking about whether it is a past weakness, present 
consideration, future enhancement, or an image of the customer's 
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environment At this point you are looking for images of the customer 
environment e.g., images of the customer using the product or doing a 
task. Later you’ll come back to the interviews and pick up the other 
customer voices which will be used to generate customer requirements. 

Write each image on a 3” x 3" Post-It Note with a black pen. Use the 
customer’s actual words or your observations. Don’t worry about 
including some statements that may also fit into another category (e.g., 
a past weakness). If in doubt, write it down. It is not unusual after all 
of the interviews are completed to have 100 or more images. 

Examples of images from the Stripping Basket Case Study are below. 



Casting into a 

CHANNEL WHERE 
THE CURRENT IS 
HEAVY. 



Having to cast 

QUICKLY. 



Rod UN DER YOUR 
ARM , STRIPPIN G 
BOTH HAN DS INTO 
THE B ASK ET. 



Tips 



• Debrief as soon as possible after each interview and use this time to 
improve your interview skills and intuition about the customer’s 
needs. 

• Remember to leave flexibility in your schedule to take advantages 
of the offers for plant tours, interviews which go longer than 
expected, etc. 

• Schedule time for the team to get together during the interview 
weeks to share observations and insights. 



Completion Checklist 

At the completion of Step 2, you should have: 

• Transcripts from each customer visit 

• Image statements written on 3x3 "Post-It” labels 
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step 3: Develop a Common 

Image of the Customer 
Environment 



The purpose of this step is for the team to create a common mental map 
of the customer’s environment. This map is the primary device for 
keeping the team grounded in the customer’s use environment. The 
Multi-stage Picking-out Method was developed by Professor Kawakita 
as part of the KJ process. It is used to select the best images collected in 
step 2 for subsequent use in creating the Image KJ. The image KJ follows 
from the work of Professor Ohfuji and colleagues, and is a critical 
component of the requirement statement development process described 
in Step 4. 



Gather Image Statement Labels 

During Step 2 , Collecting the Voice of the Customer, image statements 
identified from each interview were transcribed onto 3”x3” Post-It 
labels. These labels should be collected and posted on a large table or 
to a wall. Use the Multi-stage Picking-out Method (MPM), CQM 
Document 5P, to reduce the number of images to between 24 and 30. This 
number is a manageable size for the Image KJ. When the number of 
images increases toward 30, the time required for the KJ increases 
dramatically. 



MPM Selection Criteria 

The following criteria for selecting images for the Image KJ should be 
displayed prominently next to the MPM work area and repeated before 
the start of each round. 

1 . Images should reflect the personal experience of a user. They are 
often written in first person. For example: ”I come home from 
fishing and throw my basket by the porch stairs and there it 
stays.” 

2. Images should be capable of standing by themselves without 
amplification or explanation from the author. For example: 
"Fishing from the platform on the bow of the boat.” 

3. Diversity of images is essential. It may be better to select an image 
label of slightly inferior quality according to Criteria 1 and 2 in 
order to obtain coverage of an area not yet covered. 
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Select Most Significant Images 

MPM is a tool which involves the team in choosing the vital few from 
the useful many. The focus is on picking up strengths, not eliminating 
weaknesses. Select a target number of Image labels usually around 24, 
and a cutoff number (1/3 more than the target). The theme is usually: 
"What are the scenes or images of the customer's environment?" The 
theme and selection criteria should be jx)sted next to the labels. 

We assume familiarity with the use of MPM. See CQM Manual 5P for 
detailed instructions. 

In the initial ("multi-dot") rounds, every participant can select any and 
all the labels they want to keep by marldng a small dot on the lower 
right comer of the label with a red pen. There is no time limit to any 
round or limit to the number of labels you can mark. However, it is 
important to recognize that the number must be reduced and you need to 
be successively more selective. The initial rounds are completed when 
the number of labels remaining is approximately equal to the cutoff 
number (around 32). 

In the final ("limited-dot") rounds, you can select only a predetermined 
number of labels. This limit is usually calculated by dividing the 
target number by the number of participants. In these rounds, you select 
in order, each label being read out loud before the next in line selects. 
When everyone has selected their allotted number of labels the target 
number should have been reached and the selection process is complete. 



Create Image KJ 

The Image KJ is slightly more difficult than a traditional KJ. The KJ 
process, as outlined in the CQM KJ manual (Document 2P), provides the 
basic KJ steps. Several additional guidelines are provided for Image 
KJs: 

• The theme for the image KJ might be: "What are the scenes or 
images of the customer’s environment." 

• The scrubbing step should not change the words on the label. If a 
label requires clarification the author should provide additional 
context by referring to the appropriate interview transcript. 

• The titles to the groups should continue to reference the customer 
and their environment. Titles which contain the product as the 
subject tend to be requirement or solution oriented rather than 
descriptions of the environment. 

• There is no need to vote on the most important groupings. 
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What are the scenes or images which come to mind when you visualize Saltwater Fishing? 
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Bedford, MA 



Reflection 



Upon completion of the Image KJ, it is time for your team to pause and 
reflect upon what you have learned. It may be that you have learned 
you still need information from other customers who were excluded, or 
that you learned something using this process that may not have been 
uncovered otherwise. You should also reflect on the process and how it 
might be improved the next time. 



Tips 

• Avoid selecting Images which are highly abstract and don't give 
you a good picture of the customer's use environment. 

• Images which contain or evoke emotion are preferable to those 
which are purely descriptive. 

• Watch for a tendency to migrate from customer to product 
orientation during title development. (Titles will start looking like 
customer requirements instead of images of the customer's 
environment). 

• Be careful of clarification discussions which are not anchored in the 
interview transcripts; they can lead to misinterpretations. 



Completion Checklist 

At the end of this step, the Image KJ should be complete. 
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stage 2: Converting 

Understanding 
Into Customer 
Requirements 



The second of the five stages in Concept Engineering is Converting 
Understanding into Customer Requirements, This stage distills what 
was learned from the customer exploration into a small set of well 
understood key customer requirements. The Image KJ developed in Step 
3 will be used as a contextual anchor to ensure that the requirement 
statements developed are consistent with the customers’ environment. 

There are three steps in Stage 2. In Step 4, the transformation method 
converts the customer’s language, often laden with emotion, into a 
requirement statement better suited for use in downstream development 
activities. Step 5 uses the Multi-stage Picking-out Method to identify 
the vital few requirements from the useful many. Step 6 employs the KJ 
method to develop new insight and team consensus regarding the 
relationships among requirements. 
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Concept Engineering Stages 



Stage 2 Steps 
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step 4: Transform Voices into 
Customer Requirements 

The purpose of this step is to develop customer requirements which are 
unrestricted and unambiguous. Requirement statements should reflect 
the customer's need, not potential solutions to the need which 
inadvertently restrict the requirement. Requirement statements should 
minimize ambiguity. When requirement statements are expressed in 
ambiguous terms that allow for diverse interpretations, subsequent 
development activities can be both inefficient and ineffective. The 
methodology links each selected voice with an appropriate image or 
images from the Image KJ developed in Step 3. Key items (key 
thoughts or essential ideas) from the image-voice association are 
extracted and converted into requirement statements through an 
iterative refinement process. Translation guidelines are used to 
develop requirement statements which are unrestrictive and 
unambiguous. 
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The Customer Requirement Worksheet 



To assist you in recording Customer Voices and Images, extracting Key 
Items, and developing Customer Requirements, you use a simple 
worksheet. A small-scale example can be seen below, and a full-page 
version is included in Appendix F. Using this worksheet not only helps 
you generate requirements but provides you with an audit trail that can 
be useful for clarifying discussions about requirements. 



The Worksheet 



## 


Customer Voice 


Image 


Key Hem 


Cust. Reqt 

































The first column is simply for the purpose of numbering the voices. The 
next column comes directly from the interview notes. A "Voice" is 
defined as an individual thought, idea, or statement that is to be 
considered and can be understood on its own merits. Not every word 
that is captured in the interview notes is transcribed here. Only the 
direct language that is meaningful as a unique thought from that 
interviewee is defined as a voice. There may be 50 or more voices from a 
single interview. 

These guidelines will help you make good use of the requirements 
worksheet: 

• Begin to use the worksheets early in the process. 

• Leave blank rows between voices on the worksheet so you can 
expand the number of Key Items and Customer Requirements per 
voice. 
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Selecting Voices for Transformation 

Each interview can produce dozens of customer statements (voices) 
appropriate for transformation into customer requirement statements. 
Ideally, each voice will be transformed in order to maximize the return 
from the exploration investment. However, on occasion this will not be 
feasible. In these instances there are several alternatives for selecting 
a subset of the total voices for review. 

The preferred alternative is for every voice to be read, linked to 
appropriate images, and have key items determined. If the key item 
has been previously identified and transformed into a duplicate 
customer requirement statement, no further action on this voice is 
required. 

Another alternative is for all the voices to be reviewed and categorized 
according to common criteria, e.g., using the Net-Touching method. The 
voices which are the best exemplars of each category are then selected 
for transformation. 

Finally, another alternative is for the team to identify a list of 
screening factors for voice selection. For example, statements regarding 
regulatory requirements might be excluded if it was already known 
that all regulatory requirements would be met. Only the voices which 
passed the screening criteria would be transformed. In determining the 
screening factor list, it is important that each person selecting the 
subset of voices clearly understands the screening criteria. 



Identifying the Contextual Anchor 

Customer requirements must be related to the customer's actual 
environment. To ensure that customer requirement statements reflect 
the customer’s experience, each selected voice is associated with an 
image from the Image KJ develop>ed in step 3. The voice can be linked 
at any level of abstraction, i.e. first-, second-, or third-level titles, or to 
an actual data label; most often the linkage is made at the first-level 
title. It is often the case that a voice can be linked to more than one 
image. Creating these additional linkages is encouraged, as the new 
patterns of association can be a source of discovery. 



Extracting the Key Item 

Often it is not clear what the voice, in the context of the image, really 
means. Determining the key items from the marriage of the voice and 
image serves as a bridge to assist the development of the customer 
requirement statements. One approach to determining the key items is 
to start by systematically identi^ng significant words in the voice 
and making a corresponding key item. Another method is a free- 
association approach in which the first things that come to mind are 
used as the starting point. Key items are subjected to rapid iterative 
improvement in the requirement statement development process. 
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Creating the Requirement Statement 

The customer requirement statement must be stated in the language of 
reports without being unnecessarily restrictive or ambiguous. If the 
requirement is clear and concise it will allow later development 
activities maximum design flexibility while minimizing the risk of 
misdirected effort. 

Based on our experiences, we developed three Translation Guidelines 
and a Transformation Worksheet, App>endix (F). The guidelines are 
presented in precedence order, i.e. guideline number 1 is more important 
than guideline 2, and so on. 

Translation Guideline 1: Avoid Statements of Means 



Requirement statements that contain solutions severely restrict the 
fre^om of designers to consider alternatives. The team should work 
hard, therefore, to avoid statements of means - ”how to" language - 
when writing requirements statements. 



Customer’s Voice 



Image 



Key Item 



Requirement Statement 



"Quick release basket so it 
doesn’t get in the way 
when moving around the 
boat after a fish." 



"Fishing from the 
platform on the 
bow of a boat." 



1. The basket is 
released easily. 



(-) The basket has velcro 
fasteners. 

(+) The basket is 
releasable with one hand. 



The better (+) requirement statement clearly states the requirement 
without restricting potential solution alternatives. The weak (-) 
statement restricts the solution alternatives to velcro. 



Translation Guideline 2: Avoid Abstract Terms 

Requirement statements which contain abstract or vague terms allow 
for multiple interpretations of the customer's requirement. This can 
result in development activities which are inconsistent with the 
customer’s actual requirement or at cross-purposes with each other. 



Customer’s Voice 



Image 



Key Item 



Requirement Statement 



"Durable, material made 
out of cane won't last- 
plastic will last longer 
than me." 



"I come home from 
fishing and throw 
my basket by the 
porch stairs and 
there it stays" 



1. The basket must 
withstand the 
environment. 

2. The basket must 
last several 
seasons. 



(-) The basket is durable. 

(+) The basket is 
saltwater resistant. 

(+) The basket with- 
stands exposure to 
the sun. 



The better (+) requirement statements clearly state the requirements for 
environmental factors. In the weak (-) requirement statement, the term 
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"durable", permits many possible interpretations. In this project, 
additional requirement statements were also developed for durability 
as it relates to rough physical handling. 



Translation Guideline 3: Use multi-valued thinking 

Design constitutes a continual series of tradeoffs. Requirement 
statements which are multi-valued imply a range of performance and 
allow the developer flexibility in design. Requirement statements 
which are not multi-value oriented are "0/1," or "here/not-here" in 
orientation. Requirement statements which are written in a fashion 
which implies the requirement is either totally included or excluded 
unnecessarily restrict design freedom. 



Customer’s Voice 


Image 


Key Item 


Requirement Statement 


"How the water spills out 
of it; drainage.” 


"Tides, waves, 
seaweed” 


1. drainage 


(-) Water does not 
accumulate in the basket. 

(+) Water drains freely 
from the basket. 



The better (+) requirement statement implies a performance measure 
(time) and range of performance (quick to slow). The weak (-) 
requirement statement addresses whether water does or does not 
accumulate in the basket. 



Grammar Guidelines for Writing Requirements 
Effectively 

Professor Shiba has taught that the subject of the requirement 
statement must relate to the product or its attributes. Additionally, 
Strunk and White's classic The Elements of Style identifies some 
principles of composition that are essential to keep in mind during 
requirement statement development. 

'B« clear. When you $ay something, make sure you have said it.” 

Using a simple sentence structure, subject-verb-modifier helps you to be 
clear. For example: 

"The basket adjusts to placement on body.” 

Instead of: 

"The basket position can be adjusted to accommodate various 
casting styles. " 

‘Place emphatic words of a sentence at the end. The proper 
place in the sentence for the word or group of words that the writer 
desires to make most prominent Is usually the end.” 

For example: 



175 



"Line is tangle-free." 

Instead of: 

"Tangle-free line is essential" 

"Use the Active Voice. The active voice Is more direct and 
vigorous than the passive.” 

For example: 

"You can release the basket with one hand." 

Instead of: 

"The basket can be released with one hand." 



"Put statements In positive form. Make definite assertions.” 

For example: 

"Water drains freely," 

Instead of: 

" Water does not accumulate in the basket." 



Transformation Tips 

• Develop the requirement statement by rapidly making successive 
improvement of the key items and requirement statements rather 
than attempting to write a statement which addresses all of the 
thoughts on the first try. 




• Use of the word "not” is an indicator of a requirement statement 
which is weakness-oriented not strength-oriented. Positive- 
orientation is preferred as the absence of weaknesses is not strength. 

• Use of the words "must” and "should” usually indicate a lack of 
multi-valued orientation. 
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• Use of the word ’’and” usually indicates that more than one 
requirement is contained in a single statement. 

• It is not unusual for a single key item to produce more than one 
requirement statement or for multiple key items to generate only one 
requirement statement. 

• Use of the worksheet will create a permanent audit trail that has 
proven very valuable in subsequent clarification discussions. 

• You should block out a couple of dedicated hours for transformation 
work, and then leave it; come back to it at another time with a 
fresh perspective. Don’t force it. 

• Requirement statements can be developed by individuals or in small 
group)s. We recommend that teams work in small groups (2 or 3 
people) initially. 

• When a group works on developing custon^r requirement statements 
together, members may find that there are different 
interpretations of the key items in a voice, or different 
interpretations of the image to which the voice relates. This may 
be a result of different members having heard different comments 
from their customer interviews or simply based on their functional 
backgrounds, e.g., marketing vs. engineering. Different 
interpretations are opportunities for creativity — for finding some 
hidden requirements. Pursue all of these opportunities for 
requirement development. You will have a chance later in the 
process to validate the requirement statements with customers. 

• Occasionally an appropriate image cannot be located on the Image 
KJ. It is acceptable to use an image which did not make the final 
round of MPM , and is not on the Image KJ as the anchor. 

• Any solutions discovered in reviewing visitation transcripts should 
be identified and segregated for use in Stage 4: Concept Generation. 



Completion Checklist 

At the completion of this step all selected voices should be transformed 
into Customer Requirement statements using the Translation 
Worksheets. Well-written requirement statements should: 

• Have simple, subject-verb-modifier, sentence structure. 

• Be free of sp>ecific solutions. 

• Be at a concrete level of abstraction through the use of definite, 
specific language. 
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step 5: 



Select Most Significant 
Requirements 



The purpose of this step is to determine a manageable set of key 
customer requirements to focus on. The empathy for the customer which 
has been develo|?ed during prior steps is brought to bear on the selection 
of the most significant Customer Requirements. The Multi-stage 
Picking-out Method (MPM )will be used to select the most significant 
requirements. The requirements selected in this step will drive all 
further effort in Concept Engineering. 



Gather Customer Requirement Statements 

The final customer requirement statements which were developed in 
step 4 (last column of the worksheet) should be transferred onto 3”x3" 
"Post-It" labels and placed on a laige table or wall. Due to the large 
number of requirement statements (300 or more is not unusual) it is useful 
to collocate similar requirement statements. 

To initiate the collection process, the facilitator randomly picks one 
customer requirement statement label. It is read out loud and placed on 
the wall/ table. Without discussion, anyone else who feels he or she 
has a label which is similar hands it to the facilitator, who places 
these labels, without reading or comment, in the vicinity of the first 
label. When all offered labels are pwsted, the facilitator randomly 
selects another label reads it out loud and places it on the wall or table. 
Again, anyone who feels he or she has a similar label will have these 
posted by the facilitator without comment. This continues until all 
labels are posted to the board. No attempt is made to formally 
categorize the groupings. 



Define the Selection Criteria 

Discuss selection criteria before conducting the MPM. It is very 
important that the selection criteria be focused on the customer's 
requirements, not on solutions or features which are personally 
attractive to members of the group. Additionally, because only the 
twenty to thirty most significant requirements are ultimately selected, 
an emphasis on diversity is important. Finally, as all labels will be 
reviewed and refined in Step 6, focus on the underlying thought, not 
necessarily the words of the requirement statement. 
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Select Most Significant Customer Requirement 
Statements 



The selection process should be effective and efficient. MPM is a tool 
which involves everyone equally in systematically choosing a small 
number of items from a large number. The goal is to pick up strengths not 
to eliminate weaknesses. A target number of requirement statements is 
selected (usually around twenty-four) and a cutoff number (1/3 more 
than the target number) is also determined. The theme is written and 
displayed prominently next to the labels. The theme is usually: 

"What are the customer’s most significant requirements for 

?" The selection criteria should also be posted next to 

the labels. 

In the initial ("multi-dot") rounds, you can select any and all the labels 
you feel are significant by marking the lower right comer of the label 
with a marker. There is no time limit to any round or to the number of 
labels you can mark. However, it is important to recognize that the 
number must be reduced and that you have to be successively more 
selective. The initial rounds are completed when the number of labels 
remaining is approximately equal to the cutoff number. 

In the final ("limited-dot") rounds, you can select only a predetermined 
number of labels. This limit is usually calculated by dividing the 
target number by the number of participants. In these rounds, you select 
in order. After a label is selected, it is read to the group before the next 
person in line selects. When everyone has selected their allotted 
number of labels the target number should have been reached, and the 
selection process is completed. 



Selection Tips 

• When organizing the requirement statements at the beginning of 
the MPM, do not title the groups; omitting titles will prevent 
premature classification of requirement categories. 

• Emphasize that the MPM rejects are not thrown away and can be 
used in later development activities. This selection process is 
designed to select those key requirements which will define the 
product in the marketplace. 

• Discourage discussion during the early rounds; this prevents 
valuable energy and time from being spent on requirement labels 
that will not survive the pick-up process. 

• When discussion regarding the meaning of a label does occur, go 
back to the worksheets to ensure that the requirement statement is 
placed in the proper context of customer voice and image. 

• During the MPM it can be useful to have someone, who is not 
involved in the selection process (perhaps a facilitator) separate 
the rejected requirement labels into groups with a common theme. 
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During the final rounds of the MPM it may prove useful if the final 
rejects are kept handy for review during the ’’check for omission” 
stage of the Requirement KJ in Step 6. 



Completion Checklist 

Upon completion of this step twenty to thirty key requirements should 
have been selected for subsequent investigation. 
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step 6: Develop Insight into 

Customer Requirements 

This step forms the foundation for creative concept generation. Using 
the requirement statements selected in step 5, the essence and structure 
of the Customer Requirements are abstracted and examined using the KJ 
method. The KJ encourages the creation and examination of a myriad of 
requirement relationships, which facilitates the opportunity for 
creativity and insight. 

Review Customer Requirement Statements for 
Clarity 

This is the first px)int at which each requirement statement, one at a 
time, will be systematically reviewed by everyone. If diverse 
interpretations exist, the Transformation Worksheet should be 
reviewed to ensure consistency with the original customer’s voice and 
image. The requirement statement should be rewritten for clarity if 
required. If a everyone agrees on the interpretation of the requirement 
statement, discussion is not necessary. 



Create Relationships 

Follow the KJ process as outlined in the CQM KJ manual. The theme for 
the KJ might be: "What are the most significant customer requirements 
for ?" 



Reflect 

Before progressing to the next stage of Concept Engineering, reflect on 
what has been learned to this point. Are there any surprises or 
inconsistencies which might constitute opportunities for innovation? 
What additional information would be useful to have? What would be 
done differently if it could be done over again? Is there a need to go 
back for additional exploration before moving forward? 



Tips for Requirements KJ 

• In the check for omission step, after the first round of grouping, refer 
to the Image KJ and late-round rejects from the requirement MPM to 
help identify px)ssible key omissions. 

• The first round of grouping should take an hour or so if all of the 
plausible relationships are to be considered. Don’t rush the 
grouping. 

• The concept of taking only one step at at time up the ladder of 
abstraction must be enforced in the title making process. 
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• Titles should only address the intersection, not the union of the 
labels in the group. 



Completion Checklist 

At the completion of this step the Customer Requirements selected in 
Step 5 will be structured in a Customer Requirements KJ. 



182 



What are the Customers* Requirements for a Stripping Basket? Should Allow You to Focus on 

Fishing by Eliminating Line Problems 
^ » V and Discomfort. 
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stage 3: 

Operationalizing 
What You Have 
Learned 



We have heard repeated complaints that development concepts 
changed over time, even after managers had "signed off’ in various 
review processes. Clearly, the product development process is 
inefficient if people do not agree on what is important or expected. 
Stage 3, Operationalizing What You Have Learned, should ensure that 
the key Customer Requirements are clearly, concisely, and 
unambiguously stated in measurable terms. 

At the completion of this stage, the team will have developed a 
Quality Chart and Operational Definitions. By reviewing these 
documents, anyone associated with the development effort can easily 
see the relationships between Customer Requirements and can 
determine their relative priority. They can also find specific 
measurement techniques for assessing conformance to the requirements. 
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Concept Engineering Stages 
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step 7: Develop and Administer 
Questionnaires 



The product development team will enter Step 7 with a well-focused 
list of key requirements. The main objective of Step 7 is to help the 
design team develop a feeling for which requirements should receive 
the greatest attention from the team in later design efforts. 

We will use three procedures to achieve this goal: 

• The Kano Questionnaire, which characterizes the nature of the 
requirements 

• The Self-stated Importance Questionnaire, which measures the 
importance of the requirements 

• The Critical Requirement Questionnaire, which selects the critical 
groups of requirements. 

In general, the steps to follow for all questionnaires are as follows: 

1. Develop the questionnaires 

2. Test the questionnaires and revise if necessary 

3. Administer the questionnaires to customers 

4. Process the results 

5. Analyze the results 



This section emphasizes the Kano Questionnaire, a tool we are just 
learning to use. We briefly address the other two questionnaires first. 



Self-Stated Importance Questionnaire 

According to research by Professor Hauser at MIT, The Self-Stated 
Importance Questionnaire can be helpful in understanding the relative 
importance of each requirement for the customers. 

Method 

Constructing the Self-Stated Importance Questionnaire is straight- 
forward: 

1. For each of the requirements (the black-level labels) on the 

Requirement KJ, construct a question in the following general 
format: *"Hozv important is it or would it be if: [requirement xj?” 

For example: "How important is it or would it be if the line is cast 
without drag?" 

2. Provide a scale on which customers can mark their responses, from 
"Not at All Important", to "Extremely Important." 
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Stripping Basket Self-Stated Importance Questionnaire 



How important is it or would it be if: 
the line is cast without tangles? 


Not at All 
Important 

1 2 


Somewhat 

Important 

3 4 


Important 

5 


Very 

Important 
6 7 


Extremel) 

Importani 

8 9 


How important is it or would it be if: 
the water drains from the basket freely? 


1 


2 


3 4 


5 


6 


7 


8 


9 


How important is it or would it be if: 
the basket color does not fade over time? 


1 


2 


3 4 


5 


6 


7 


8 


9 


How important is it or would it be if: 
the basket can be attached at different 
body positions? 


1 


2 


3 4 


5 


6 


7 


8 


9 



Critical Requirements Questionnaire 

As we indicated in the discussion of the Requirements KJ, we reconamend 
having customers perform the voting step. If you wish to have 
customers vote on your Requirements KJ, it is convenient to gather their 
input at the same time you distribute the other questionnaires. 
Therefore, you may also want to construct a Critical Requirements 
Questionnaire. The only drawback is potentially overloading your 
customers with materials. You must make a judgement call about how 
much material they can handle. 

Method 

1. Make a list of the red-level labels (black labels that are lone 
wolves at the red-level belong in the list too). If you wish, you can 
provide additional detail by indenting the text from the black- 
level groups under the appropriate titles. 

2. Instruct the customer to pick the 3 most important items from the 
red-level list and rank them in priority. 



Kano Questionnaire 

Professor Noriaki Kano has developed a tool which helps us 
understand the relationship between the fulfillment (or non- 
fulfillment) of a requirement and the satisfaction or dissatisfaction 
experienced by the customer. Kano achieves this by classifying each 
customer requirement into one of 5 categories: must-be, attractive, one- 
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dimensional, indifferent, and reverse. The definitions for these terms 
and a synopsis of the underlying theory are presented in Appendix D, 
"Understanding Kano Theory." We suggest that first-time readers 
review this appendix before proceeding. 

The Kano Questionnaire is only one tool for setting development 
priorities. While it provides a unique perspective on customer 
requirements, it is probably most effective when interpreted as a guide 
to be combined with other requirement assessnnent methods. 

Method 

In constructing the questionnaire you will form two questions for each of 
the requirements appearing in your Requirements KJ. The first question 
will always refer to a situation in which the requirement is met, and 
will be worded in a format similar to the following: "If the [product] 
satisfied [requirement x], how would you feel?” This is call^ a 
functioning question. The second question will always refer to the case 
where the requirement is not met. This is called the dys functioning 
question, and is worded in a format similar to the following: "If the 
[product] did not satisfy [requirement x], how would you feel?" 

Developing the Kano Questionnaire 

Divide the requirements from your KJ among team members.Write a 
functioning and dysfunctioning question for each requirement using the 
guidelines below: 

• You may have to step down the ladder of abstraction to construct a 
clear question. Avoid straying from the original intent of the 
customer requirement statement. Refer to the Requirement KJ and 
Translation Worksheets if necessary. 

• Beware of polar wording in the question pairs; multi-valued 
orientation is preferred. Consider this functional question: "If line 
placed in the basket stays in it until cast, how would you feel?" 
instead of wording the dysfunctioning question : "If line placed in 
the basket falls out before casting, how would you feel?"; It would 
be preferable to ask, "If some line placed in the basket falls out 
before casting, how would you feel?" 

• . Don’t try to cram several thoughts into one question. You want to 
know which question the respondent is answering. If a particular 
requirement contains more than one thought, use more than one 
Kano question. If you opt to generate more than one question for 
some requirements, you should keep in mind the need to keep the 
survey as short as possible. 

• When you have functional and dysfunctional wordings for each 
question, put the five standard answers beside each question as 
follows. 
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Stripping Basket Kano Questionnaire 



8a.If the line does not move 
around in the basket, how do 
you feel? 


1 . I like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it that way. 


8b.If the line moves around in the 
basket, how do you feel? 


1. I like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it that way. 


9a.If line placed in the basket 
stays there until casting, how do 
you feel? 


1. I like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it that way. 


9b.If some line placed in the 
basket comes out before casting, 
how do you feel? 


1. I like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it that way. 


lOa.If line in the basket is tangle 
free, how do you feel? 


1. I like it that way. 

2. It must be that way. 

3. I am neutral. 

4. I can live with it that way. 

5. I dislike it that way. 


lOb.If line in the basket 
occasionally tangles, how do you 
feel? 


1. I like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it that way. 


Half line gathers naturally in the 
bottom of the basket, how do you 
feel? 


1 . 1 like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it that way. 


1 Ib.If line does not gather 
naturally in the bottom of the 
basket, how do you feel? 


1 . I like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it that way. 



Testing the Questionnaires 

Since the questionnaires will be sent to many customers, it's important 
that they be understandable. We recommend testing all of the 
questionnaires internally before distributing them to customers. A test 
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run will help identify confusing wording, typographical errors, or 

unclear instructions. 

Refining the questionnaires may require iteration. These guidelines can 

help you test your questionnaires effectively: 

1. Have the team answer the questionnaire first. Think of a customer 
and try to predict their responses. Note which questions you think 
your customer may not understand. 

2. Select f>eople inside your company to answer the questionnaire, and 
administer it to them. Select from a variety of backgrounds, e.g. 
senior managers, development engineers, marketing personnel, etc. 

3. Revise the questions and retest. 

Administering the Questionnaires 

There are many details to consider in administering the questionnaires. 

• Select the customers you would like to survey. We suggest returning 
to the customer selection matrix to develop a target list, applying 
the same criteria discussed in that step. In order to assure a 
statistically significant sample, most teams poll considerably more 
customers than were interviewed. Remember that not all customers 
will respond (for more information, see Appendix C, "Additional 
Hints on Administering Surveys"). 

• Decide on the medium to be used. You might use telephone (voice or 
fax), face-to-face presentation, mail, or other means. The most 
common method is through the mail. If you plan to use the mail, 
write a letter of introduction which explains the purpose of the 
survey and includes directions for the customer. See Appendix C for 
an example. 

• Collect demographic data which will enable you to distinguish 
potential market segments if they exist. Consider collecting 
information on company and personal characteristics, familiarity 
or experience with product, use of competitors products, etc. 

• Include instructions for filling out the questionnaires. This is 
particularly important for the Kano questionnaire since it is likely 
to be new to customers. 

• If you are using the Self-Stated (Questionnaire in addition to the 
Kano (Questionnaire, use the same sequencing of questions to make 
comparing the results of the two questionnaires easier. 

• Send out the surveys. Keep a log of customers to whom the surveys 
were sent, along with the date. This will help to avoid 
duplication if you choose to expand your sample later, and to follow 
up if necessary. 

• Record responses as they arrive. 



191 



Processing the Kano Results 

1. Translate each response to the question pair into a quality element 
code. To do so, look at each pair of questions on the Kano survey, 
and note the Functioning and Dys functioning answers for each 
question. Refer to the Two-dimensional table of evaluation, shown 
below, and locate the cell at the intersection of the Functioning and 
Dysfunctioning responses. 



Two-dimensional table of Evaluation 
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like 
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must-be 
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5. dislike 


R 


R 


R 


R 


Q 



Customer Requirement is: 

A: Attractive O: One-dimensional 

M: Must-be Q: Questionable result 

R: Reverse I: Indifferent 



To illustrate how to score the questionnaire, question 9 from the 
stripping basket questionnaire is shown below. 



9a. If line placed in the basket 
stays there until casting, how do 
you feel? 


1. I like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it. 


9b.If some line placed in the 
basket comes out before casting, 
how do you feel? 


1 . I like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it. 



If the customer respx)nded to part (a) by circling #1 "I like it that 
way" and responded to part (b) by circling #5 "I dislike it", then 
look at the intersection of the first row and the fifth column, you 
will find an ”0,” indicating that the customer views the amount of 
line which falls out of the basket as a One-dimensional element. 

2. Record the requirement codes (i.e.. A, M, O, R, Q, or I) on the 
tabulation matrix. An example of the tabulation matrix appears 
below. 
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3. Repeat the above steps for all the questions on the survey for each 
customer who returned the survey. 

4. In the far right column of the tabulation matrix, assign a grade to 
each of the requirements. The grade should be whichever code 
appears most often in the responses for that requirement (i.e., it is 
the mode of the responses). the next section and Appendix D for 
more sophisticated analysis of Kano results. 



Evaluations of CR. for Stripping 
Basket Kano Questionnaire 
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Kano Response Analysis 

Several benefits are obtained from analyzing Kano data: 



• Gaining a better understanding of requirements 
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• Prioritizing requirements for development activities 

• Distinguishing market segment characteristics. 

In the discussion of the underlying theory. Appendix D, we addressed 
the qualitative distinctions between the five types of quality elements. 
Some screening rules were provided in the previous section. This section 
will focus on techniques we can use to interpret the data for prioritizing 
future development activities. Remember that the purpose of these 
questionnaires is to better understand the characteristics of the 
customers* requirements. The responses to these questionnaires should 
be seen only as a guide — no exact answer is provided as to which 
features must be included in the product, or which requirements need not 
be fully satisfied. 

Alternative Analysis Approaches 

A simple way to rank the requirements is to score each according to the 
mode, the most frequently occurring dimension in each row of the 
tabulation matrix. Thus, a requirement voted Must-be by 43% of 
respondents and Attractive by 38% of respondents would be interpreted 
as a Must-be requirement. 

You may want to investigate the respx)nse distribution beyond the 
simple mode, i.e., looking at the second most frequent dimension for 
each requirement. For example, take a case where there are two 
questions and fifty responses to each. Thirty customers rate the first 
requirement Attractive and twenty rate it Indifferent. On the second 
requirement, thirty customers again rate it Attractive, but the 
remaining twenty rate it Must-Be. In this case, it is likely that the two 
requirements should be treated differently by the team. The second 
requirement should receive higher priority from the team. 

We suggest constructing a spreadsheet with columns for the first, 
second, and third most frequent responses. The next step is to rearrange 
the rows into grouf)s according to the following order: Must-Be's first, 
One-Dimensionals second, followed by Attractives, Indifferents, and 
Reverse requirements. 

The Self-Stated Questionnaire responses can be helpful in further 
sorting the Kano responses. One way to order the requirements within 
each group is by importance ranking. For example, if there were 15 
requirements whose mode was ’'Attractive,** you might use the Self- 
Stated Importance data to rank the Attractive requirements in 
descending order of importance. This would help to further 
discriminate which Attractive requirements the team might want to 
focus on. 



Interpretation 

Decisions on what will be included in your product are influenced by 
many factors. As a general guideline, we suggest that you seek to 
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fulfill all Must-Be Requirements, be competitive with market leaders 
on the One-Dimensional Requirements, and include some 
differentiating Attractive elements. 

In general, the return you can expect from fulfilling a requirement (in 
terms of customer satisfaction) should guide the effort you invest to 
fulfill it. Improving performance on a Must-Be Customer Requirement 
which is already at a satisfactory level is not productive when 
compared to improving performance on a One-Dimensional or 
Attractive Requirement. Qassifying Customer Requirements into 
Kano’s dimensions can help you to focus on the vital few where the most 
leverage exists. 

Additionally, you should use demographic data in conjunction with 
questionnaire analysis to identify potentially differentiating market 
segment characteristics. 

Continuous/Graphical Analysis Approach 

The Statistics Subcommittee of the CQM Research Committee 
considered more sophisticated methods of questionnaire analysis. 

These ideas are presented in Appendix D. 



• The time you are taking to hear your customers’ views contributes to 
the company’s professional image. The format of the 
questionnaires should further enhance that image. 

• Listen carefully and without bias to what your internal test 
customers say. If they find something confusing, it is quite likely 
that others will as well. Revise questions or add instructions as 
necessary. 

• Establishing the method by which the data will be analyzed 
before disseminating the questionnaires will save time. Will you 
use manual or automated input and analysis tools? Knowing U^is 
will enable you to marshal the necessary resources while you are 
waiting for replies. 

• When two Kano codes are tied in the scoring for a given question, 
consider: 

1. Following up with customers for additional insight 

2. Looking for market segmentation differences 

3. Selecting the classification that would have the greatest 
impact on the product (use the following ordering: Must-be 
>One-dimensional >Attractive >Indifferent) 

• It is helpful to get some advice from someone in your firm who has 
experience with customer surveys. 

• A small gift to those who complete the questionnaire may improve 
response rate and generate customer satisfaction! 
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Completion Checklist 

At the end of this step, you should have: 

• Compiled and tested questionnaires 

• A nuiling list of respondents 

• An analysis template. 
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STEP 8: Generate Metrics for 
Requirements 



This step will identify the metrics which will be used to assess each 
Customer Requirement. You will use these metrics to objectively 
evaluate alternative design solutions in the Concept Selection Stage of 
Concept Engineering. Metrics can also be used to benchmark 
competitive products. Additionally, the process of generating metrics 
increases your understanding of requirements by requiring you to work 
down the ladder of abstraction. 

The team will brainstorm possible metrics for each requirement and 
then select the smallest set of metrics (usually one or two) which cover 
the specific requirement. Some requirements are relatively straight- 
forward (e.g., ’’the line in the basket is tangle free when cast "). 
Developing a measurement unit to assess these requirements is fairly 
uncomplicated. 

However, some requirements are more ambiguous (e.g., ’’the basket is 
comfortable"). Developing measurement units for these requirements 
can be difficult, in that any one measurement unit will measure only 
part of the requirement and in addition will also assess some other 
dimension. In these instances multiple metrics may be requried to assess 
one requirement. 

At the end of this step, a tree diagram will be used to organize the set 
of metrics and identify any missing ones. 



Method 

Brainstorm Customor Requiromant Metrics 

Working with one Customer Requirement at a time, use traditional 
brainstorming techniques to generate a list of possible metrics which 
could be used to assess this requirement. Try to make this list 
collectively exhaustive; attempt to get all ideas for measuring each 
requirement surfaced for consideration. Develop a brief working 
definition for each {X)ssible metric. This definition need only be 
detailed enough to ensure that members of the team have consistent 
interpretations of the metrics’ intent. 

The team can split into small groups or pairs and divide up the 
Customer Requirements to save time. If the team is going to split up, we 
recommend working on one or two Customer Requirements as a full group 
first so that everyone gets a sense of how this is done, and then dividing 
the remaining requirements among teams of two or three members. 
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Evaluate Validity 



Determine how valid each metric is; validity is the degree to which 
each metric actually measures the requirement, i e., directly measures 
the requirement rather than measuring something else. Informal 
validity assessments can be made through group consensus rating of 
each potential metric on a high, medium, or low scale. With informal 
assessment, a simple thumbs up for high, thumbs across for medium and 
thumbs down for low vote has worked well. With this approach, select 
the score which receives the most votes. In the event of a tie, select the 
lower value, i.e., select low if there is a tie between medium and low. 

Alternatively, the team could defer to the opinion of a recognized 
expert. Formal validity assessments also can be conducted using 
statistical procedures such as correlation analysis or signal-to- noise 
ratios. 

Evaluate Feasibility 

Feasibility is how easy or difficult it would be to use the metric; to 
actually collect and interpret the data. 

Informally assess the feasibility of using each metric using a high, 
nr^edium and low scale. 

As a guideline in evaluating feasibility, consider briefly describing 
how the data would be collected ( local check sheets, existing reports, 
etc.), and how the data would be analyzed,( e.g., personal computer 
spreadsheet analysis or mainframe application development). You 
might use this technique for every metric, or when the team has 
difficulty assessing feasibility or disagrees on the feasibility of a 
metric. 

Assess Ambiguity 

In order to select the smallest set of metrics, you need to assess the 
ambiguity of each Customer Requirement. Requirements that are 
ambiguous will probably need more than one metric to assess them well. 
Give each Customer Requirement a rating on an ambiguity scale such as 
the following: 



Everyone Possibilities Everyone 

on team A difference for multiple on team 

interprets interpretation interpretations agrees on 



differently 



exists on team 



exist on team 



meaning 



1 



2 3 4 5 6 7 
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Select Vital Few Metrics 



For those Requirements with ambiguity ratings in the 5-7 range, select 
the best metric to carry forward to the next step. If you have a highly 
valid, highly feasible metric, then the choice is obvious. If not, use the 
ranking scale below. 

For those Requirements with an ambiguity rating of less than 5 , you’ll 
probably need to select more than one metric. Two or three will 
probably be enough to cover the requirement. Select the highest 
ranking metrics which seem to cover the different asp>ects of the 
requirement. 



Validity 


0 


© 


O 


O 


© 


O 


A 


A 


A 


Feasibility 


0 


O 


© 


O 


A 


A 


© 


o 


A 


Rank 


1 


2 


3 


4 


5 


6 


7 


8 


9 



© = Strong O = Medium = Weak 



R»poat for oach Cu»tom«r R»qulr»m«nt 

Repeat the process of generating metrics, assessing validity, 
feasibility and ambiguity and selecting the smallest set for each of the 
CRs. 



Stripping Basket Example 

C.R: Line is cast without tangles. 



Feasibility 


Validity 


Measure 


A 


A 


Count the number of perpendicular loop)S before cast. 


A 


O 


Count the number of overlapping loops. 


© 


© 


Calculate the percentage of fouled casts 


© 


© 


Calculate the percentage of fouled drops. 



Ambiguity assessment; 6 (everyone basically agrees on interpretation; 
it is straightforward). One metric is selected to carry forward to the 
tree diagram: Calculate the percentage of fouled drops. 
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Construct a Tree Diagram of tho solected Motiics 

A Tree Diagram is an analytical tool used for assuring a complete set of 
answers to a "How to" question. The Tree Diagram is often used for 
solution generation, as in 'Tiow do we successfully complete Project X". 
In this step of Concept Engineering, we use the Tree Diagram to answer 
the question, "How do we measure the key Customer Requirements for 
Product X?". This tool systematically builds a hierarchy of metrics and 
their purposes, and by doing so a team can then step down this 
hierarchy and search for missing or better metrics or groups of metrics. 
An example from the stripping basket is on page 3.18. 

1. Prepare a Tree Diagram (CQM manual 4P) using the theme: "How 
are the customer’s requirements measured?" Use the metrics carried 
forward from earlier work as the first-level labels for the tree. 
Leave much room on the paper for adding new metrics or groups of 
metrics in later steps. 

2. Group each metric by common purpose. Ask, "What's the purpose of 
collecting this data?" 

3. Write a title for each group which completes the sentence, "the 

purpose of using these metrics is to measure ." 




4. Continue the process of grouping by similar purposes and writing 
titles until there are five or fewer groups. 

Top-down chocking 

This stage of the Tree Diagram method is quite important, and too often 
teams rush through it and don't gain the benefit. It is a structured 
brainstorming approach to generating better metrics. Take your time 
and work at generating missing or better metrics. 

1. Start at the top of the tree and work down one level at a time, 
asking, "what, if anything is missing?" For example, if after 
grouping was completed there were three high-level groups of 
metrics, check if these groups assess the full set of customer 
requirements or if something is missing. If you identify a missing 
group, write a title for that group at the proper level of abstraction 
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and then work down the tree, creating titles and metrics as 
appropriate. 

2. Step down to the next level on the tree, continuing to search for 
omissions. 

3. At the first title level, continue top-down checking, improving 
existing metrics or adding new ones. 

Evaluating the metrics 

Once the tofxiown checking is complete, it is time to evaluate the full 

set of metrics in order to identify the strongest ones. 

1 . Draw an evaluation table on the bottom of the tree which consists 
of one row each for effectiveness, feasibility, and rank, and one 
column for each metric. 

2. Assess effectiveness of each metric. Ask, "how effective is this 
metric toward achieving the purpose of the title above?" You will 
most likely use high, medium, and low consensus ratings. 

3. Assess feasibility of each metric. Either use the feasibility rating 
you gave the metric in the brainstorming step, or if the metric is a 
new one which emerged from the top-down checking, then assess 
the feasibility the same way you did earlier. 

4. Rank the metrics using the same table used to select metrics prior to 
the tree diagram. 



Tips 

• It is highly likely that the most effective metrics are not measures 
which are collected and analyzed in the current information 
system. Therefore, don’t limit brainstorming to only measures 
currently collected. 

• If it is necessary to find stronger metrics, return to the tree diagram 
and look for metrics which are highly effective, but medium or low 
in feasibility. Think about ways to make these metrics more 
feasible. 

• This may be a good time to bring others into the team who have 
more knowledge about metrics commonly used to assess customer 
requirements. Balance this against the time necessary to get others 
familiar with the requirements and the process. 

• When in doubt about a particular high/medium/low assessment of 
validity or feasibility, err on the low side so that the strongest 
metrics are easier to identify. 



Completion checklist 

At the end of this step, you should have: 

• Worksheets that document metrics development 
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A tree diagram of metrics that covers the full set of Customer 
Requirements. 



stripping Basket Tree Diagram Example 



Check if basket prevents 
tangles/snags from occurring. 



Check if basket 
prevents line 
movement in 
bottom of 



Measure how line 
covers the basket 
bottom 



Count 


Measure 


Measure 


Measure 


Measure 


Count 


the# 


the height 


how much 


outer edge 


the 


the 


of loopjs 


of loops 


line falls 


droop in 


distribut- 


number 


over the 


from 


out of 


four wear 
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of loops 


top of 


bottom of 


basket 


positions. 


loofTS in 


in each 


the 


the 


when 




basket 


section. 


cones/ 


basket. 


placed in 




bottom. 




stales. 




it. 









Effectiveness 
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A 
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© 


Feasibility 
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7 


2 
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1 
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STEP 9: Integrate Understanding 

About Customer Requirements 

This step captures the knowledge gained to date in the development 
process about the customer's requirements and their metrics. It displays 
the information in a format which allows for clear, concise 
communication to everyone concerned. It is an important reference point 
for dowmstream development activities. 

This step ends Stage 3 of Concept Engineering and in a sense finishes the 
work in the "requirements space." The next stage. Concept Generation, 
begins work in the "solution space." 



Method 

Prepare the Quality Chart 



1 . Convert the Customer Requirement KJ into a tree structure and 
rewrite all of the labels onto 2"x3" post-its. 



1st level 


2nd level 


3rd level 






Line is stationary 
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CO© 
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in basket 


E 


© © ^ 
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1— o © 


Line placed in the basket 
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a. 




stays there until cast 


w 

c 

</> 
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© f 


Line in the basket 


c 

(D 


S-s® 


is tangle free 


> 


® 3 

3 © 




© 




w 

Q. 


*50© 
cr © -^ 

mo© 

S E « 


Basket naturally gathers 


© 


line in the bottom 


© 

© 


S 85 




cn 




Line is cast 






without drag 



2. Place the CR Tree on the left vertical axis of the matrix. 

3. Place the entire Metrics Tree Diagram on the top horizontal axis of 
the matrix. 
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4. Leave room for two columns just to the right of the requirements for 
the results of the importance and Kano analysis. 

5. Add one row at the bottom starting, it just to the right of the two 
columns from Step 3, for noting the operational definition 
identification number. 

6. Draw a grid using the lowest level labels on each axis to determine 
the width of the rows and columns. 



Examplo: Quality Chart 




Assess the strength of Requirement to Metric relationship 

1. Start with metrics which were ranked ”1” from the evaluation of 
the tree diagram. 

2. One metric at a time, assess the correlation to each requirement. 

Ask, "How well is requirement measured by metic?" 

3. Rate each metric as to how well it assesses each requirement on a 
high, medium, or low scale , or blank for no correlation. 
Alternatively, the team could defer to the opinion of an expert. 

4. Continue with each metric which was rated a "1". Check for 
requirements which don’t have a strong correlation to a metric 
(there will most likely be some). If there are requirements without 
a strong metric, go onto metrics rated a "2", and assess each of these 
against the requirements. Check again for requirements which are 
not covered. 
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5. Continue correlation assessment until all requirements have at least 
one strong metric or until you’ve assessed all metrics. 

6. Formal assessments can be conducted using statistical procedures for 
assessing correlation or signal-to-noise ratios. This type of 
assessment would be time-consuming and probably best used in 
situations where the team feels their insight is cloudy, or the team 
feels the need to calibrate its intuition. 



Evaluate the Matrix 

Select the best set of metrics by following these guidelines: 

• If there are any rows for recpiirements without a strong relationship 
to at least one metric, identify this as an area that needs further 
metrics generation. 

• If there are any rows for ambiguous requirements without strong 
relationships to at least two variables, identify this as an area 
that needs further metrics analysis or generation. 

• If there are any requirements covered by an excessive number of 
metrics, investigate the potential to eliminate metrics. 

• If a metric is correlated with many requirements (i.e., 4 or more), 
this is an indication that the metric is highly abstract and it may 
be difficult to interpret. YouTl need to break the metric into 
elements which relate to different requirements. You can use the 
operational definitions which are described next to further define 
the metric so that it directly measures a requirement 

Develop Operational Definitions for Selected Metrics 

Op>erational Definitions provide detailed plans for how requirements 

will be assessed. Create an operational definition for each selected 

metric, using the following outline: 

• Define the unit of measure 

• Define where the data will be collected 

• Define when the data will be collected 

• Define what specifically will be observed 

• Define how the data will be collected 

• Define how the data will be displayed 

• Define who is responsible for what. 

Finally, test the Operational Definitions by trying to use them. It is 

highly likely that they will require revision after initial trial. 
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Example of an Operational Definition 



OPERATIONAL DEHNITION FOR DROP TIME 
Unit of measure: seconds 

Location: landing #6 on the building E-52 fire escape 

Wrap the first 30' of line (tapered end) around a tube sock and secure 
vsrith two windings of duct tape. 

Strip off enough line (^55') from the reel for the sock to just reach the 
ground when dropped off the landing. 

Strip off another 10* of line from the reel. 

Randomly determine the order of basket configurations to be tested. 
For each configuration: 

-drop the sock over the side; 

-strip the line into the basket without looking; 

-before dropping the sock, look in the basket and assess: 

-the -distribution of line in the bottom of the basket; 

-the -% of line above the top of the cones; and 
-the -number of perpendicular loops; 

Have the timer drop the sock, starting the stop watch when they let 
go and stopping the time when the so^ strikes the bottom; 

Record the time it took for the sock to drop as well as any 
observations regarding line payout, such as tangles, excess line 
payout, or others, on the attached check sheet. 

Rep>eat the process four times for each configuration. 



Reflect 

This is an important time for the team to stop and reflect. You’ve come 
a long way since Step 1, Plan for Exploration. Think about what you've 
done and what you've learned. What insight has been gained? What 
surprises did you discover? What additional information do you need? 
What would you do differently next time? 



Tips 

• Don't stretch the correlation analysis. There should be many blank 
cells (indicating no correlation) on the (Quality Chart. 

• It nnay not be possible to cover every requirement with a highly 
correlated metric; do the best you can. If the requirement is one of 
critical importance, additional effort at developing a highly valid 
metric may be necessary. 

• Quality Charts can be huge, unwieldy wall charts. One way to 
make the chart more manageable is to rewrite the Customer 
Requirements KJ and Metrics tree diagram labels onto onto 2"x3" 
post-its with the 2” side up against the axis for the correlation 
assessment. It will make the chart easier to work with. 
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The Quality Chart can be used as the basis for conducting 
competitive Benchmarking 



Completion Checklist 

At the completion of this step you should have: 

• A Quality Chart which includes the Customer Requirements, 
metrics and correlation assessment 

• Operational Definitions for selected metrics. 
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stage 4: 

Generating 

Concepts 



This stage marks the transition in the development team's thinking 
from the "requirement or problem space" to the "solution space." This is 
the stage of Concept Engineering that many development teams 
describe as the opportunity to "finally have some real fun.” 

Many of us have been trained to arrive at solutions as quickly as 
possible; good job performance has been, somewhat erroneously, 
equated with quick fixes. Conversely, the first three stages of Concept 
Engineering teach the virtue of patience as applied to product 
development: accurately define requirements before generating 
solutions. 

In Stage 4, the same disciplined thinking process is applied to 
generating jx)tential solutions. The team will systematically 
decompose the development objectives and then cycle between 
independent and group activities to generate and identify the strongest 
solutions. 

The self-documenting nature of Concept Engineering is maintained 
throughout Stage 4, where ideas and combinations of ideas are 
completely preserved for reflection and later review if necessary. 
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Concept Engineering Stages 



Stage 1 ; Understanding the 
Customer's Environment 



Stage 2: Converting 
Understanding into Customer 
Requirements 



Stage 4 Steps 



Stage 3: Operationally 
Defining Requirements for 
Downstream Development 



i 




V 


Stage 4: 

Generating Concepts 


J 






Step 1 1 : 

Generate Ideas 

J 












V 


Stage 5; 

Selecting Concepts 


J 




f' 


Step 1 2: 

Generate Solutions ^ 
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In Stage 4 it is desirable to quickly generate many diverse concepts. In 
Stage 5 you will rapidly focus on the most promising solutions and 
converge on the dominant ones. This process is illustrated in the figure 
below. 




Stage 4, Generating Concepts, comprises three general steps, as 
described below. 



Step 10, Decompx)se the Problem, sets out to reduce the complexity of 
the problem by breaking it down into more easily handled subproblems. 
Many different types of decompositions of the same design problem are 
encouraged, to facilitate better coverage of the potential design space. 
Decomposition also allows for individuals or groups to work in parallel 
to develop solution ideas for different comjX>nents of the product or 
system. 

During Step 11, Generate Ideas, ideas are rapidly and exhaustively 
brainstormed and improved for each of the subproblen\s. 
Simultaneously, a search for existing solutions is also conducted in order 
to assure complete coverage of the solution space. Lastly, the most 
promising ideas are picked up by group consensus. 

Step 12, Generate Solutions, begins with an exhaustive enumeration of 
solution combinations (combinations of solutions to each of the 
subproblems) followed by a rapid enhancement of the best solutions. 

The solution enhancement is accomplished primarily by the group’s 
collective intuition, perhaps aided by some laboratory testing or 
experiments. The output is a set of plausible concepts (solution 
combinations) which serve as input to Stage 5, Concept Selection. 
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step 10: Decompose Design 

Problems 



The purpose of this step is to divide the complex design problem into 
smaller, but independent, subproblems. Multiple diverse 
decompositions are encouraged as an aid for enhancing coverage of the 
potential solution space. You can deepen and broaden your insight from 
the generation of ideas if the design problem is decomposed in diverse 
ways. 



Development Objective 

The first step in decomposing the design problem is to state the 
Development Objective. The Development Objective defines the 
direction of your design efforts. It is a clear, concise articulation of the 
requirements which focuses concept generation. 

Often it is adequate to use the title or the conclusion from the Customer 
Requirements KJ to serve as the basis of the Development Objective. 

For example, the stripping basket Customer Requirements KJ conclusion 
reads: ”It should Allow You to Focus on Fishing by Eliminating Line 
Problems and Discomfort.” A translation of the conclusion to a 
Development Objective is: ”The Stripping Basket must allow the 
customer to focus on fishing by eliminating line problems and 
discomfort.” 

Write your development objective on a flip-chart paper and hang it 
conspicuously on a wall to remind the team of this focal point. 



Decomposition 

Decompose the problem into its subcomponents. You can dothis in many 
ways. We recommend trying 3 or 4 decomposition possibilities and 
documenting them on flip-chart paper. Start with your organization's 
traditional development decomposition first, that is, how you would 
normally split up the design task. Examples of possible decompositions 
follow; 

• Decompose by technical disciplines (mechanical, electrical,etc.) or 
functional components (e.g., inputs, processors, outputs). 
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• Customer Requirements KJ. Decomposing using custonier 
requirements provides a customer-oriented perspective. 
Decomposition by first- and second-level titles is recommended. 

• Metrics Tree Diagram. Decomposition by metrics from the Tree 
Diagram provides a technical perspective. Decomposition by first- 
and second-level titles is reconrumended. 

• A process flow describes the secjuence of actions involved before, 
during, and after the use of the product or service. 

Select a minimum of two decompositions for futher development. One 
will usually be your traditional approach and the other should be more 
customer-oriented. 



Coupling 

Coupling allows you to identify problem areas which are independent 
of each other, in which solutions in one area do not effect solutions in 
another area. When the development team judges that the solution 
ideas of two (or more) decomposed subproblems overlap, those problems 
may be linked, or coupled, together. For example, in the stripping 
basket the line tangling and line drag subproblems are not independent; 
one is connected to the other. These requirements can be coupled 
together and solutions can be generated for both subproblems at the 
same time. 

Examples of the Stripping Basket decomposition style are provided 
below. You will notice in the examples which follow that "coupled" 
subproblems are separated by heavy bold lines from the adjacent 
subproblem. 

The decomposition format lists the subproblem as the column heading 
and space below in which to eventually add the brainstormed ideas for 
solutions(Step 11). Use large format flip-chart paper to document your 
decompositions. You'll use the same sheets for Step 11, Idea 
Generation. 
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The first example of coupling is the Stripping basket Customer 
Requirements KJ diagram. This example shows that the next higher 
level title is included in the table to promote clarity to the team. 



Basket prevents 
line problems 






n 1 . 


1 


The line moves only 
when desired. 


When required, line 
comes out of the basket 
easily. 


Basket accommodates 
casting, stripping, and 
movement. 









Stripping basket Metrics Tree Diagram. This example shows that the 
next higher level title is included in the table to promote clarity to the 
team. 




Problem Decomposition Example — Stripping Basket Process Flow. 



Attaching basket 
to wearer 


Stripping line into 
the basket 


Casting line from 
the basket 


Detaching the 
basket from the 
wearer 
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Tips 

• Work from the Quality Chart develo|:>€d in Step 9 when 
developing the customer-oriented decompositions. 

• Stick with it. You may have difficulty with decomposition; good 
decoupling will pay off later when it will be easier to combine 
solutions from different subproblems. 

• Don’t be concerned if your team finds that the decomposition by 
functional breakdown or by process flow doesn't allow for coupling 
of subproblems — couples may not exist! 

• Before moving forward to Step 11, post the Problem Statement 
sheet on a conspicuous wall to remind the development team of the 
task at hand. 

• In order to leave adequate room for the activities of Step 11, use one 
sheet of paper per subproblem or coupled subproblem. Don't be 
alarmed at the stack of flip-chart sheets your team will generate 
during Step 10! 



Completion Checklist 

• A dearly articulated Development Objective 

• At least two development decompostitions 

• Decomposed, independent, subproblems written on large sheets of 
paper. 
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step 1 1 : Generate Ideas 

Step 11 forms the bridge between the requirenr^ents space of the previous 
steps and the solutions space. Ideas are brainstormed for solutions to 
the subproblems and are combined with existing concepts found through 
researching literature and talking with experts and customers. This 
process should generate an extensive, nearly exhaustive list of px)ssible 
solutions to consider. 



Researching Old Concepts 

In order to identify existing solutions, consider the following resources: 

• Expert analysis 

• Database searches 

• Competitive benchmarking 

• Solutions gleaned from your customer interviews. 

Expiert analysis may be provided by consultants, universities, R & D 
laboratories and your customers. Database resources, such as j^atent 
searches, on-line periodicals and trade or industry associations may 
produce valuable ideas. You can gain insight by evaluating the "Best In 
Qass" or ’’Best In Industry" products. 

You may choose to split the team up into individuals or pairs to do this 
research. For example, one pair could travel to the library to conduct a 
patent search, another could identify and interview industry experts, 
and still another could identify |X)tential competitive products for 
subsequent team evaluation. 

The results of your research must be clearly documented and assembled 
as a part of the project documentation. 



Generating New Concepts 

The generation of new ideas is a process of unconstrained thinking 
followed by structured reflection. The unconstrained thinking expands 
the boundaries of ideas for evaluation; the structured reflection focuses 
on picking up the strengths and opportunities contained in each idea. 
Initial ideas are seldom the best; push yourself and the team to create 
extensions and hybrids of these early ideas. 
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Expand feasibi# design spoce 

Individually, or in a small group, create an exhaustive list of solution 
ideas focusing on one subproblem at a time. 

For each subproblem identified in Step 10, brainstorm ideas which 
expand the boundaries of ordinary idea generation. For example, for 
each subproblem, offer ideas which test: 

• Market constraints. Propjose ideas that will flop in the 
marketplace. 

• Technology constraints. Propose ideas achievable only with 
alchemy. 

• Organizational constraints. Propose ideas likely to get you fired. 

Rapidly generate feasible solution ideas which will cover the 
expanded solution space. 

• During this independent generation of ideas, members should view 
the subproblems from each of the product’s stakeholder 
perspectives: end-user, dealer, salesperson, installer, etc. 

• Solution ideas do not have to be at a specific level of abstraction; 
any idea can contain strengths which can serve as a springboard to 
an even stronger idea. 

• Record each idea on a 3"x3" Post-It and place on the Subproblem 
decomposition worksheet prej>ared in Step 10. 

A portion of an "Idea Generation” sheet for the Stripping Basket 
subproblem "Line moves only when desired" is shown on the next page. 

Group collaboration 

Meet as a team and: 

• Logically classify and group the previously generated ideas, 
writing titles for each group. This is similar to the Net-Touching 
process outlined on page 1-10. 

• Review the independent idea generation results, focusing on 
enhancing the strength of the brainstormed ideas, not identifying 
their weaknesses. For each idea listed, ask "what are the 
strengths associated with this idea?" New ideas are added to the 
list as developed. 

• Have another session (10 minutes) of indep>endent idea generation; 
push team members to identify additional ideas. Ideas are posted 
to the sheet on 3"x3" Post-Its. 

• Review the new ideas with a focus on enhancing their strengths. 
For each idea listed, ask "what are the strengths associated with 
this idea?" New ideas are added to the list as developed. 
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• Quickly assess the feasibility of the posted ideas. 

• Select the most promising feasible ideas for each subproblem for 
further development. 

Several methods for selecting the most promising ideas have been used: 

• MPM, after discussing the strengths of each idea, to select the best 
three to five ideas. (Some teams go straight to final round 
selection, giving each member only one vote.) 

• Apply DeBono's PMI process. PMI is an attention directing tool. 
Frst note the positive or Plus aspects of an idea, then note the Minus 
and then note the Interesting points of an idea over a period of 
about 2-3 minutes. A PMI worksheet can be found in Appendix (F). 

• An intuitive or consensus decision process by the team. 



Stripping Basket Idea Generation Examples 



Line moves only when desired 



Ideas which would flop 

• flat smooth bottom 

• plain flexible bottom 

Ideas achievable only with alchemy 

• electrostatically charged line 

• magnetically charged line 

Ideas sure to get me fired 

• spinning reel(real fly fishermen don’t 
use spinning reels) 

Other ideas 

• astro turf 

• monofilament stakes 

• monofiliment loops 

• one traffic cone 

• nails 

• duct tape 
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• Start idea generation with the customer-oriented decomposition 
first. Complete your traditional decomp>osition after all 
alternative views have been explored. 

• Suzzane Merrit of Polaroid’s Creativity Lab suggests the use of a 
’’Mental Excursion” as one method for generating additional ideas. 
Use the Image KJ to take a ’’trip” into the customer’s world while 
concentrating on a subproblem. 

• Setting specific time limits for idea generation can help 
concentration. 

• Plan on several cycles of individual generation and group 
collaboration. Planning several short sessions per week has proven 
useful for some teams. 

• Schedule a day for the team to absorb the results of the research on 
old concepts. The presentations from the research should become 
part of the project documentation. 

• The first attempts at generating ideas through brainstorming can be 
carried out as a team activity. After the team gets the hang of 
brainstorming in this fashion, subproblems may be allocated to 
individuals or pairs. 

• Personal criticism and competition should be strongly discouraged. 

• Do not forget to add the ideas you uncovered in your customer visits 
to the list of ideas for evaluation. 



Completion Checklist 

• Subproblems and lists of possible solutions are documented on sheets 
of large chart paper; the most promising ideas are highlighted. 

• Notes on research of existing concepts. 
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step 12: Generate Solutions 

This is the final step in Concept Generation and it is characterized by 
creativity and reflection as you identify the strongest solution concepts 
from the many ideas generated. Rapid and exhaustive linkages, 
combining ideas from different columns, are made to create solution 
concepts. The use of the summary table maintains the self-documenting 
nature of Concept Engineering by providing an audit trail of the range of 
ideas and idea combinations generated. The strongest solution concepts 
are carried into Stage 5 for more detailed analysis. 



Create a Summary Table for Each 

Decomposition Style 

1 . Collect each of the sheets of chart paper containing subproblems 
and their group-selected solutions. Keep them in their 
decomposition groups. 

2. Within a decomposition group, tap>e the sheets together side-by- 
side, creating one long document. For exaunple, if you decompose 
using three methods, you will now generate three summary tables 



Create Solution Concepts 

1 . Considering only one sununary table (and therefore one 
decomposition approach) at a time, link together subproblem 
solutions into a total concept. In other words, select items from each 
column and combine them to create a solution concept. 

2. Ideally, create as many alternative combinations as possible, 
thereby providing an opportunity for insight and learning. An 
alternative would be for each team member individually to create 
the solution concept they feel is strongest. 

3. Document solution concepts by recording each combination of 
subproblem solutions you generate on a Concept Description Sheet- 
a description of the solution concept in one page or less . 



Select the Strongest Solution Concepts 

Review each Solution Concept individually. Eliminate those that are 
apparently infeasible or can readily be discounted by group consensus. 
However, expert evaluation and laboratory testing may be required in 
order to judge the relative strength of many of the other combinations. 
Therefore, the end point of Step 12 and the start point of Step 13 is 
defined as the need for the development team to use resources beyond its 
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own intuition in order to accomplish the convergence on the best solution 
concepts. 



Tips 

• You'll need long walls in order to accommodate the summary tables; 
plan to use appropriately large facilities for Step 12. 

• When creating solution concepts don't be too critical; focus on 
generating many solution concepts. 



Completion Checklist 

• Summary tables for each decomp)osition method. 

• Concept Description sheets for each concept to be carried forward to 
Stage 5, Concept Selection. 

• Concept Description sheets for each elinninated concept (save these 
to maintain the completeness of the project documentation 
package). 
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stage 5: Selecting 

the Concept 



In this final stage of Concept Engineering, a product concept is selected. 
In the previous stage, the development team generated a wide array of 
solutions to collectively address the set of Customer Requirements. In 
this stage, the solutions act as raw material to be refined, combined, 
and evaluated in an iterative fashion until a small number of complete, 
superior solutions remain. The best concepts are subjected to a final 
evaluation, combining numerical scoring algorithms with the 
considerable intuition built by the development team during its efforts. 
Selection of the dominant concepts consistent with market requirements, 
company capabilities, and company philosophy will be the final result 
of this step. 

In Step 13, Screen Solutions, the team will think individually and 
together, seek expert help, and exp>eriment in the laboratory in an 
iterative process of combining and improving initial solution concepts 
to develop a small number of sup>erior concepts. In Step 14, Select the 
Solution, the "surviving" complete concepts are evaluated in detail, 
using returned Kano and Importance rating data. Two separate 
nunverical algorithms can be used to generate scores for each concept. In 
Step 15, Reflect on the Process, the numerical "scores" of the concepts 
are considered in conjunction with other decision factors, to select the 
dominant concepts. 
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Concept Engineering Stages 
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step 13: Screen Solutions 

Prior to this step, the team has developed a large number of informally 
screened solutions which have been documented on Concept Description 
sheets. The task of this step is to screen, combine, and improve this set 
of solutions until the best elements have been fused into a small number 
of complete concepts. The tools at hand for this task are: 

• Creative brainpower 

• Experimental work 

• A Concept Screening Matrix. 

At the completion of this step, the remaining mature concepts will be 
ready for final evaluation, to be handled in the next step. Concept 
Selection. 



Screening Process 

The screening process is best explained with the help of a simple flow 
chart. 




The initial input is the set of selected Concept Description Sheets from 
Step 12, Solution Generation. Some initial solutions may be incomplete, 
addressing only a fraction of requirement space. As iterations of this 
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process are completed, these solutions will address an increasingly 
larger fraction of requirement space. (The Concept Description Sheets 
must be updated accordingly.) Eventually, there will be only a few 
concepts remaining, but these will be complete solutions addressing all 
of the requirement space. When these completed concepts are 
optimized to the satisfaction of the team, Elution Screening is 
complete, and Solution Selection, Step 14 , may begin. 



Evaluate Solutions against Requirement Space 

The term Requirement Space is useful because it suggests the image of a 
map containing the set of customer requirements, along with their 
quality dimension and relative imjx)rtance. To improve a set of partial 
solutions, there must be some way to evaluate them against this map. 

Screening Matrix 

One way to perform such an evaluation is by evaluating each solution 
(Concept Description) against the set of Customer Requirements 
generated from the Requirements KJ. The resulting screening matrix 
looks something like: 



Solutions 





A 


B 


C 


D 


E 


F 


G 


H 


I 


t 

OS 

j 




C.R. 1 


+ 1 


+2 


0 


N/A 


N/A 


+1 


0 


+1 


-2 


-1 




CR.2 


0 


+1 


N/A 


0 


+1 


0 


+2 


-1 


0 


0 




CR.3 


+2 


0 


+1 


0 


-1 


+2 


+ 1 


N/A 


0 


+1 




CRA 


-1 


N/A 


N/A 


+2 


-1 


-1 


+ 1 


-2 


N/A 


+ 1 




CR.5 


N/A 


+1 


+1 


-1 


0 


N/A 


N/A 


N/A 


+2 


-2 




CR.6 


N/A 


+2 


0 


0 


0 


-1 


-2 


+1 


+2 


-1 




CR.1 


N/A 


N/A 


0 


+1 


-1 


+2 


-1 


0 


+1 


“4/A 





The solutions are arrayed along the horizontal axis and the Customer 
Requirements are arrayed along the vertical axis. Each cell contains an 
evaluation of how well a particular solution satisfies a given Customer 
Requirement. 
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R«fer«nc« Concopt 

Because the numerical ratings in the screening matrix are relative, 
there must be some reference to compare against. Select a reference 
product. A natural selection might be your current product or a 
competitor's product that you deem "best in class." You can either 
assign the rating 0 to each facet of your reference selection or you can 
evaluate your reference selection, treating 0 as 'neither a market 
advantage nor a market disadvantage'. In either case, evaluate your 
solutions in reference to the rating you gave to your reference concept. 

Rating Seal* 

The example shows evaluations including -2, -1, 0, 1, 2 and N/A. The 
higher ratings (1 and 2) reflect superior performance, 0 reflects some 
chosen reference performance, and -2 and -1 reflect sub-reference 
performance. Some teams find the use of -, 0, + provides sufficient 
information about relative performance. The N/A stands for not 
addressed, which simply means that the solution is not complete and 
does not account for that requirement. As more iterations occur, the 
number of N/As in the screening matrix will decrease, and the final 
concepts at the end of Solution Screening should address all 
requirements. 

Alt«rnaHv« Scraening Matrix 

The Solution vs. Customer Requirements screening matrix is not the only 
useful screerung matrix. To m^e judgments about how well customer 
requirements are satisfied by a solution can be difficult because it's 
hard to measure customer requirements directly. 

The set of Metrics generated by the team to measure customer 
requirements can be easier to use to assess the solution concepts. The 
complete set of metrics should cover the same requirement space as the 
Requirement KJ. Therefore, it should be as valid to compare solutions 
against metrics from the Metric Tree Diagram as it is to compare 
solutions against requirements with an alternative screening matrix 
like the following one. 
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Solutions 





A 


B 


C 


D 


E 


F 


G 


H 


1 


OS 

J 




Metric 1 


+1 


+2 


0 


N/A 


N/A 


+ 1 


0 


+ 1 


-2 


-1 




Metric 2 


0 


+1 


N/A 


0 


+1 


0 


+2 


-1 


0 


0 




Metric 3 


+2 


0 


+1 


0 


-1 


+2 


+1 


N/A 


0 


+1 




Metric 4 


-1 


N/A 


N/A 


+2 


-1 


-1 


+1 


-2 


N/A 


+1 




Metric 5 


N/A 


+1 


+1 


-1 


0 


N/A 


N/A 


N/A 


+2 


-2 




Metric 6 


N/A 


+2 


0 


0 


0 


-1 


-2 


+1 


+2 


-1 




Metric 7 


N/A 


N/A 


0 


+ 1 


-1 


+2 


-1 


0 


+ 1 


N/A 





An added advantage with metrics is that you n\ay be able to conduct 
relatively quick and simple experiments to determine solutions' 
effectiveness. 

You can use either or both screening methods. Keep in mind that these 
tools are decision aids: use the approach most appropriate to your 
situation. 

Completing the Screening Matrix 

Completing the Screening Matrix may involve experimentation. The 
ratings for some cells may be easy to learn or already known. Others, 
however, may require physical modification of devices, quick and dirty 
prototype mockups, or computer modeling. The idea here is not to jump 
full spe^ into development, but to do just enough lab work to ascertain 
the feasibility and performance of particular solutions (a large number 
of mini-development cycles). 



Generate New Solution Concepts 

After evaluating solutions against the requirement space, you may 
discover that your solutions do a good job in some areas and not in 
others. Create new solution concepts by combining the strengths of 
different existing solutions. 

After completing your first Screening Matrix, talk about each solution 
in turn with the entire group. Discuss the merits and drawbacks of each 
solution. Talk about possible ways to improve solutions and, wherever 
possible, consider combining positive elements from multiple solutions 
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into a single solution. Document the new Solution Concepts and 
complete another Screening Matrix. 

After discussion, private thought, evaluation and experimentation, 
ideas have been refined and combined to create a new set of solutions. If 
the team is satisfied that the concepts are complete and there is little 
to be gained by further iterations, proceed to Step 14, Solution 
Selection. Otherwise, take your u^ated solution set and go through 
another cycle of solution generation and screening. 



Tips 

• Alternate group discussion with individual time for reflection. 

• For early screening iterations, the scale may be more 

resolution than is necessary. If you'd rather disdngwsh only 
neutral, and + for early cycles, feel free to do so. However, for later 
iterations, where you are busy fine-tuning the concept, the added 
resolution will be very helpful. 

• Reference concepts are more difficult to generate for completely new 
products. If there is nothing in existence that does what you want 
to do, try to find a set of different products that between them meet 
your customer requirements. Your reference concept will be a hybrid 
of aspects from several products. 

• Don't worry if your questiorxnaues aren’t back before you begin 
solution screening. They are unimportant in the early iterations, 
increasingly helpful in later iterations, and absolutely essential in 
Solution Selection. As your concepts become more complete, it 
becomes more important to consider the Kano dimension and the 
relative importance weighting of your requirements. These 
distinctions will significantly influence the solution "scores" in the 
next step. 

• Feel free to add new solution ideas as inspiration strikes. If they 
are appropriate, they will survive the improvement cycles to 
follow. If not, no harm done. 

• Some teams have found screening with Metrics to be easier than 
screening with Requirements, because metrics are inherently 
quantitative. 

• Make sure group discussions of solutions are frequent. There are 
several benefits. Updated solutions will converge to completed 
concepts faster if everyone develops an intuitive feeling for the 
direction other team members are heading, and can think about 
solution interaction. With frequent communication, you increase 
the tendency for the entire team to reach intuitive consensus on 
final concepts. 
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• You may find it helpful to add organizational requirements to the 
screening matrix. For example, resource availability, time, 
manufacturing capability, technological risk, etc., could be useful 
criteria for screening concepts. 



Completion Checklist 

At the end of this step, complete solutions, which address the entire 
requirement space (custonrter requirements or metrics), should be 
documented on Concept Etescription Sheets. 
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step 14: Select the Solution 

Solution Selection is a detailed nunnerical analysis and scoring of the 
nnature concepts developed during Solution Screening. You will use the 
results of these analyses in selecting a product concepts. The basic 
prenaise of evaluating solutions against the requirement space is the 
same as in the previous step. However, this final numerical analysis 
explicitly accounts for the results of Kano data and Self-stated 
Importance ratings. This scoring takes into account that some 
requirements are more important than others. 

As in solution screening, two separate Selection matrices are presented, 
one based on customer requirements, the other based on metrics. One or 
both of these matrices may be used to give the team a final detailed 
numerical evaluation of their concepts. 
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Process for Selection 

Here is the process flow for Solution Selection: 



Mature Concepts 




Once again, you ntay choose to assess your solution concepts against 
Customer Requirements or Metrics or both. The Metric algorithm, as 
before, has the advantage of being directly measurable. However, the 
Requirement algorithm is easily adaptable to Kano data. Each will be 
described and an example shown. 



Requirement-Based Scoring 

This scoring method can consider the importance weights and/or the 
Kano results for the set of Customer Requirements. 

Screening Matrix Without Kano Results 

Begin with the screening matrix. Customer Requirements against 
Solutions, (cell-performance ratings are -2,-l,0,l, and 2 as in the 
previous step), with an extra column for Self-Stated Importance 
C^estionnaire data. 
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Self-Stated 

Importance 

Ratings 


A 


Solut 

B 


ons - - 

c 


D 


CJl. 1 


1 


+2 


+1 


0 


+1 


C.R.2 


2 


0 


+1 


-2 


+2 


C.R.3 


4 


+1 


0 


+2 


+ 1 


C.R.4 


3 


-1 


+1 


+2 


0 


Solution Scores 


3 


6 


12 


9 



Scoring Without Kano 



If we included Importance Ratings and not Kano data, we would 
calculate solution scores as the sum of performance ratings, weighted by 
the corresponding Self-Stated Importance Ratings. Solution A, for 
example, would have a score of (+2)(1) + (0)(2) + (+1)(4) + (-1)(3) = 3. 
Those solutions with the highest scores would dominate. 

Screening Matrix Including Kano results: 

To account for Kano results, we must redefine the cell performance 
ratings. For example, if a given requirement has purely attractive 
quality (A) and it is implemented well, customers are very satisfied; if 
implemented poorly or not at all, customers don’t care. Suppose we had 
a solution which scored a -2 (very poor implementation) for this 
requirement. With our existing scoring method, the -2 rating would 
detract from the overall effectiveness score for the concept. However, 
we know for Attractive quality, a poor score has no effect on how a 
product is perceived. Thus, the -2 rating for Attractive quality should 
be changed to a 0 rating. Continuing this line of thinking, the various 
Kano results should be redefined as follows: 



A 


O 


M 


I 


Old -2-1012 


-2-1012 


-2-1012 


-2-1012 


New 00012 


-2-1012 


-2-1000 


0 0000 


Here is a Screening Matrix with Kano data included: 
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Kano 

Dimension 


Self-Stated 

Importance 

Ratings 


A 


Solut 

B 


ons • - 

c 


D 


C.R. 1 


M:600;40 


1 


+2 


+1 


0 


+1 


C.R.2 


0:100 


2 


0 


+1 


-2 


+2 


C.R.3 


A:50 0:50 


4 


+1 


0 


+2 


+1 


C.R.4 


M:60 1:40 


3 


-1 


+ 1 


+2 


0 


Solution Scores 











In this example, 100 people responded to Kano surveys and the team 
considered the top two I^o scores versus just the mode response. Using 
our new ratings, accounting for Kano Dinnensions, a scoring example for 
Solution A and CR. 1 follows: 

The cell performance rating is +2, 60/100 resjx>ndents chose M, and 
40/100 respondents chose O, the redefined performance rating is 
(60/100) (0) + (40/100K+2) = +0.8. This is then multiplied by the Self- 
Stated Importance Rating. 

An example of a complete solution scoring using this method follows: 





Kano 

Dimension 


Self-Slated 

Importance 

Ratings 


A 


Solut 

B 


ons -- 

c 


D 


CR. 1 


M:600:40 


1 


+2 


+1 


0 


+ 1 


C.R.2 


0:100 


2 


0 


+ 1 


-2 


+2 


C.R.3 


A:500:50 


4 


+1 


0 


+2 


+1 


C.R.4 


M:60 1:40 


3 


-1 


+1 


+2 


0 


Solution Scores 


3.0 


2.4 


4.0 


8.4 



Solution A Score = ((60/100)(0) + (40/100)(+2)l 

+ I(100/100)(0)l 

+ ((50/100K+1) + (50/100)(+1)] x4 
+ ((60/100K-1) + (40/100)(0)1 x3 



xl 

x2 



= 3.0 
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Metric- Based Scoring 

This scoring method numerically assesses solution concepts but excludes 
Kano results. Fill out the metric-based Screening Matrix (cell-scores are 
-2,-1 Al/^nd 2 as in the previous step), leaving an extra column for 
Metric Importance weights. 





Metric 

Importance 

Weight 


A 


Solut 

B 


ons - - 

c 


D 


Metric 1 




+2 


+ 1 


0 


+ 1 


Metric 2 




0 


+ 1 


-2 


+2 


Metric 3 




+1 


0 


+2 


+1 


Metric 4 




-1 


+1 


+2 


0 


Solution Scores 











Determine Metric importance Weight 

The questionnaires in Step 7 allow you to determine an importance 
rating for each requirement. In this step, you convert that information 
into Metric importance weights by combining the correlation assessment 
from the Quality Chart (Step 9) with the Self-Stated Importance 
results (Step 7). This is a proxy measure of how strongly each metric 
relates to the performance of the entire product from the fX)int of view 
of the customer. As an example, sup?pose we had the following 
Correlation Matrix: 





Self-Stated 

Importance 

Ratings 


1 


- Met 
2 


ICS - - 

3 


4 


C.R. 1 


1 


o 






o 


CR.2 


2 






o 




C.R.3 


4 


© 






o 


C.R.4 


3 


A 


o 






Metric Weighting 


17 


6 


4 


10 



Correlation Weights: © 

o 

A 



= High (3) 

= Medium (2) 
= Lx)w(l) 
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For each metric, multiply each correlation by the Importance Rating of 
the corresponding requirement. (The Self-Stated Importance Rating is 
the average from all returned surveys for each customer requirement.) 
The sum of these products is the Metric importance weighting. 

For Mebic 1, there is a medium correlation with C.R. 1 (correlation 
factor = 2), and the Self-Stated Importance Rating = 5, a high 
correlation with C.R. 3 (correlation factor = 3) and Self-Stated 
Importance Rating = 4, and a low correlation with C.R. 4 (correlation 
factor =1), and Self-Stated Imp>ortance Rating = 1. 

Weighting for Metric 1 = (2)(1) + (3)(4) + (1)(3) = 17 

Repeat the calculation for each metric and record the Metric 
Importance on the screening matrix. 





Metric 

Importance 

Weight 


A 


Solut 

B 


ons - " 

c 


D 


Metric 1 


17 


+2 


+1 


0 


+1 


Metric 2 


6 


0 


+1 


-2 


+2 


Metric 3 


4 


+1 


0 


+2 


+1 


Metric 4 


10 


-1 


+ 1 


+2 


0 


Solution Scores 











Calculat* Solution Scores 

We now have all the information we need to score each solution. A 
solution's score will be the sum of its cell-performance ratings 
multiplied by the appropriate Metric Importance Weights. 

In our example above, the score for Solution A is 

(+2)(17) + (0)(6) + (+1)(4) + (-D(IO) = 28. 

Here is the screening matrix complete with scores ... 
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Metric 

Importance 

Weight 


A 


Solut 

B 


ons - - 

c 


D 


Metric 1 


17 


+2 


+ 1 


0 


+ 1 


Metric 2 


6 


0 


+1 


-2 


+2 


Metric 3 


4 


+ 1 


0 


+2 


+1 


Metric 4 


10 


-1 


+1 


+2 


0 


Solution Scores 


28 


33 


16 


33 



Solutions B and D are tied with the highest scores. Their strengths lie 
in different areas, but their total effectiveness is rated the same by this 
scoring algorithm. This represents an opportunity to create improved 
solution concepts through combination. 



Tips 

• It is not necessary to do both types of screening outlined in this step. 

• The creation of additional solution concepts in this stage should be 
encouraged. 

• Don't blindly follow the scoring stejss; think about what makes 
sense based on your team’s assessment. .You may find 0, + is 
satisfactory 

Completion Checklist 

At the completion of this step, a Solution Selection Matrix should be 

completed for every solution concept which received serious 

consideration. 



Step 15: Reflect on the Process 

Your team is at the final step of Concept Engineering, where you will 
make a decision on a product concept that you will present and defend 
when asking for resources to support development. At this step you 
should also reflect on the entire Concept Engineering process. 



Decision Factors 

There are many factors to consider in choosing the product concept: 
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Solution Scores 



The team has collectively constructed solutions on paper, evaluated 
their performance vs. customer requirements, and scored them with 
appropriate weighting for Kano and Self-stated Importance Rating 
Data. These scores, developed by one or both scoring methods, should 
indicate the dominant concepts. 

intuitive Convergence 

After the entire team has heard the same 'Voices of the customer," 
mapped out the requirement space, explored the solution space, and 
studied the Kano and Imfx>rtance data, it is inevitable and beneficial 
that the development team form an opinion about which solution is the 
best concept. This is another basis for nuking the decision. 

Company Capabilities 

Manufacturing capabilities, distribution channels, resource constraints, 
and product family considerations are all factors which should be 
considered in the selection decision. 

Company Philosophy 

For a product to be approved for development, it must also be consistent 
with company philosophy. It is p>ossible for solutions which meet 
nurket needs not to be consistent with company philosophy. Thus, 
company philosophy is an imjX)rtant factor to consider in making a 
final decision. 



Final Concept Decision 

Do not blindly select the concept with the highest score. Reflect on 
what you've learned. Consider factors which will ensure acceptance 
and support within the company and the distribution channel. The 
goal is to get your product into the nurket. The best product concept, if 
implemented poorly or not at all, will not nuke money. 



Concept Presentation 

You are ready to get commitment to the development of your product 
concept. Your team should have all the documentation needed to 
persuade senior management to support development of the product 
concept. 



Concept Engineering Process Reflection 

As in other TQM methods. Concept Engineering is not finished until the 
team reflects on its work and thinks about improvements to the process. 
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Reflect on how far the team has come, how much has been learned 
about the customers and their requirements. Think about the process, 
what worked well, what was difficult, what could be improved the 
next time. Document these thoughts and p>ass them along to others in 
the organization and to the CQM so that other companies can benefit 
through mutual learning. 



Tips 

• If your intuition and scores don't match, follow the data trail 
backwards from solution scoring. Pay close attention to the points 
where your intuition disagrees with the highest scoring solutions. 
This may be a clue to an error or omission along the way. When 
these final differences, if any, are ironed out, the team must commit 
to a single concept that they will support and defend. 

• If there are challenges and objections to your concept, you have a 
complete self-documented audit trail showing how you arrived at 
your final solution, along with a very large and defensible list of 
things not to do. 

• Comments on Concept Engineering can be sent to the CQM or to Gary 
Burchill at (617) 258-5586 or Diane Shen at (617) 873-3730. 



Completion Checklist 

At the completion of this step you should have commitment from your 
Development and Management team to proceed to detailed 
specification of the selected concept and notes from the team’s 
reflection on the process. 
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Appendix A 



Glossary 



Concept Engineering: a process for determining custonr^rs’ key 
requirements, creating a measurement plan for assessing compliance 
with the requirements, and developing a strong product or service 
concept which satisfies the requirements. Concept Engineering has 5 
stages: Understanding the Customer’s Environment, Converting 
Understanding into Customer Requirements, Operationally Defining 
Customer Requirements, Generating Solutions, and Selecting a Solution. 

Concept Description Sheets: a one-page or less description of a 
combination of ideas which make up a solution concept; developed at 
the end of Stage 4, and used in Stage 5, Selecting the Concept. 

Correlation Matrix: one of the Seven Management and Planning Tools 
used to look at the relationship between two sets of items; the Quality 
Chart in Stage 3 of Concept Engineering is a correlation matrix which 
examines the relationship between Customer Requirements and metrics 
in order to choose the smallest set of metrics which covers the set of 
Customer Requirements. 

Customer Requirements Statements: the outcome of translating the 
voice of the customer into requirements; a descriptive sentence stating a 
customer need, not a solution. 

Customer Requirements Worksheet a four-coliunn sheet for use in 
translating the Voice of the Customer into Customer Requirement 
Statements in Step 4 of Concept Engineering. 

Customer Voices: the first column on the Customer Requirement 
Worksheet; the actual words of the customer; each customer voice is a 
complete thought. 

Decomposition: the first step in Stage 4, Concept Generation, in which 
the team breaks the problem or objective into components by using the 
Requirements KJ, Metrics Tree, process flow, or other methods; solutions 
will be generated for each segment and then combined to form whole 
product concepts. 

Image: a mental picture of an asf>ect of the customer’s environment 
collected from a customer visit, either from something a customer said 
or from an observation of their environment; the second column in the 
Customer Requirements Worksheet, used to link a customer voice to an 
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aspect of the customer’s context to help in the interpretation of the 
voice. 

Image KJ: a KJ of a collection of images, or scenes, about the customer’s 
environment; a common mental model of the custonrter’s context in which 
the team’s product or service will be used. 

Kano Survey: a custon>er research tool used to learn more about a set of 
Customer Requirements; developed by Noriaki Kano this questionnaire 
is used to determine requirement dimensions, one-dimensional (more 
functionality leads to more customer satisfaction, less functionality 
leads to less satisfaction). Attractive (if present, leads to satisfaction, 
if not present customer is neutral), and Must-Be (if functional, customer 
is neutral, if not functional customer is greatly dissatisfied). 

Key Items: the third column on the Customer Requirements Worksheet; 
the main thoughts or key ideas from the combination of the Customer 
Voice and the image; a bridge from the fuzzy voice of the customer to 
the clear, concise Customer Requirement. 

Ladder of Abstraction: a semantics concept described by S.I. 

Hayakawa; levels at which we select characteristics to describe an 
object of our experience; lower levels on the ladder contain more 
characteristics of one object, terms on higher levels contain fewer 
characteristics of one object to describe what is common among many 
objects; Hems lower on the ladder are more factual, items higher on the 
ladder are more conceptual. 

Lead Users: a term used by Eric von Hippel to describe users of a 
product or service who have certain characteristics which set them 
ap>art and make them good targets for product development teams to 
work with in developing product concepts; among these characteristics 
are prototype experience, a strong need for a solution to the problems 
they are experiencing, and an ability to articulate the problem and 
possible solutions. 

Market-In: a concept introduced by Professor Shiba; work is a means 
not an end, the end is customer satisfaction and everyone has a customer. 

Mental Excursions: a technique for generating solution ideas in which 
you concentrate on the area of focus while reflecting on the Image KJ of 
the customer’s environment 

Metric Scoring : a scoring method of numerically assessing solution 
concepts in Stage 5 which combines the correlation assessment from the 
(Quality Chart with the Self-Stated Importance ratings. 

MPM (Multi-stage Picking-out Method): one of the methods associated 
with KJ developed by Jiro Kawakita; a tool by which a team can 
systematically reduce a large amount of data to the vital few items. 
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Net-Touching: one of the methods developed by Jiro Kawakita, a tool 
for sorting and classifying language data; organization is done logically 
as compared to intuitively in the KJ method; Unlike KJ, Net-touching 
does not lead to new insight but instead helps to quickly give some 
order to a set of language data. 

Operational Definition of a Customer Requirement: a description of a 
metric including how the data will be collected and displayed. 

Elus - Minus - Interesting: an approach for systematically exploring the 
strengths and weaknesses of a solution idea. 

Quality Chart a correlation matrix used in stage 3 to assess the 
metrics against the customer requirements in order to choose the 
smallest set of metrics which will cover the set of requirements; also 
called the "first house of quality.” 

Reference Concept usedinStageS, Step 13, Screen Solutions. In order 
to score each potential solution a reference solution is chosen for 
comparison; the reference solution concept can be the '^:>est in class," the 
cxirrent product, or a competitor’s product. 

Requirement Importance rating : a way of incorp>orating the Self- 
Stated ratings into Step 14, Select the Solution; an average of all 
returned Self-Stated surveys. 

Screening matrix: a correlation matrix used in Stage 5, Selecting the 
Concept; it compares solutions against either requirements or metrics 
and by using a reference concept, scores each of the solutions. 

Self-Stated Importance Questionnaire: a customer survey tool; 
customers give a score ranking the importance of each requirement; can 
be used in conjunction with the Kano Survey. 

Solution Concepts: ideas for product or service compx>nents are combined 
to form complete solution ideas, or solution concepts. 

Swim in Shallow Water developed from Shiba’s swimming in the 
fishbowl concept; a term used to describe practice customer visits with 
internal or friendly customers before conducting external visits. 

Voice of the Customer a verbatim transcript of the actual words of 
the customer collected in a face-to face visit or telephone call. 

WV Model: the systematic problem-solving model developed by Jiro 
Kawakita and Professor Shiba, which describes problem-solving as a 
series of steps alternating between the level of thought and the level of 
experience; contains the 7 step reactive problem solving method. 
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Appendix C: 

Additional Hints 
on Administering 
Surveys 



Response Rate 

Survey respxDnse rates can vary all over the board. For busy 
professionals such as doctors, resp)onse rates to mass mailings can be as 
low as one p>ercent. Take heart, though. Generally, resp>onse rates are 
much higher among customers (and particularly those you have 
visited). 

You will probably see lower respx)nse rates among former customers and 
prosp)ects. Market research firms often maintain statistics on response 
rates sorted by profession. Such information may be useful in planning 
the size of the mailing you need to meet your target response. 

Exp>erience suggests that 95% of those who will respx)nd at all will do so 
in three to four weeks. If you don’t have your target response by then, 
you should either follow up or do a supplenventary mailing. 

Tips to Improvo rosponso rate 

• First Class pxDStage 

• Personalized letter 

• Incentives 

• Post card warning prior to sending questionnaire 

• Post card follow-ups 
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• Professional app>earance to survey. This could include any or all of 
the following: 

20 lb. stock, smooth, non-slick paper, matching envelopes, colors 
(white, off-white, very light gray, beige, olive green ,or 
cream), black ink, same font throughout, boxes to highlight 
sections, hand signature in blue ink, standard size 8 1/2" x 11”, 
flat 8 1/2" X 11" envelope to deliver questionnaire (more 
professional, attracts attention), clean and simple layout, full 
use of white space, use of graphics if warranted, booklet form, 
shading, page numbers in upper right comer, "Please continue on 
page x" at bottom of page if warranted, note at end thanking 
respx^ndent, title printed at top of each page, color code by 
target group if necessary. 



Suggestions for the Cover Letter 

The cover letter should address the following questions: 

• What this is about? 

• Who wants to know? 

• Why do they want to know? 

• Why was I picked? 

• How important is this? 

• Will this be difficult? 

• How long will this take? 

• Will it cost me anything? 

• How will this be used? 

• Will I be identified? 

• What's in it for me (benefits, incentive, token of appreciation)? 

• When should I do it (deadline)? 
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Example Letter of Introduction 



Dear fcompanyl customer. 

At [company] we recognize that your input must play a vital role if we are to develop products 
that meet your needs. We are writing, therefore, to solicit your views regarding (product) 
features and capabilities with the intention of using your input to enhance (product) or to 
develop new products to better meet your needs. 

The enclosed questionnaire is the result of an effort by the (product) development team to apply 
Total Quality Management methodologies to investigate user requirements in the area of 
(product category) . As part of the method for determining user requirements, we visited a 
representative sample of (product) customers and non-customers, and we recorded their 
comments regarding the types of tasks they are faced with, the problems they encounter and so 
on. We have analyzed these data and developed some conclusions about (product category) 
features and capabilities that users like yourself require, and we would like to check the 
validity of our conclusions by having you answer some questions for us. 

Although some of the questions we ask may appear redundant, they were all carefully chosen 
to gather the information we feel we need to guide us in designing and developing quality 
(product category) that meets your needs. The survey should take about thirty minutes to fill 
out. Please return the completed questionnaire in the enclosed, self-addressed envelope by xxx. 

Sincerely, 

XXXXX 

(Product) Development Manager 

P.S.: To thank you for your time, a (product) T-shirt will be sent to each respondent who 
completes and returns the enclosed questionnaire. The questionnaire includes a space for you to 
indicate your preferred size. 
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Appendix D 



Kano 



Understanding Kano Theory 



Kano analysis helps us understand the relationship between the 
fulfillment (or non-fulfillment) of a requirement and the satisfaction or 
dissatisfaction experienced by the customer. Working with social 
science theories on satisfaction developed by Frederick Herzberg, Kano 
discovered that the relationship between fulfillment of a need and the 
satisfaction or dissatisfaction experienced is not necessarily linear 
(although this was the prevailing assumption at the time). He found 
that he could sort requirements into distinct classes, and that each class 
would exhibit a different relationship with respect to satisfaction. 

Kano’s theory sorts customer requirements into five categories 
(described below). A sixth category, "(Questionable Result," indicates a 
potential problem with the questionnaire. A brief definition for each 
of Kano’s dimensions is listed below: 

A Attractive Quality Elements : Customer Requirements which create 
satisfaction when fulfilled, but are accepted as is even when not 
fulfilled. In other words, the customer is greatly satisfied when 
this element is present but experiences no dissatisfaction when it is 
not present. In the stripping basket case, the ability to wear the 
basket in different positions was rated Attractive. 

0 One-Dimensional Quality Elements: Requirements which result in 
rising satisfaction the more they are fulfilled, but lead to 
increasing dissatisfaction when less fulfilled. There is a 
proportional relationship between functionality and satisfaction. 
The water drainage rate is an example of a One dimensional 
element for the stripping basket. 

M Must-Be Quality Elements : Requirements which do not lead to 
satisfaction when fulfilled but cause dissatisfaction when not 
fulfilled. Tangle-free casting was a Must-be element for the 
stripping basket. 

1 Indifferent Quality Elements : Requirements which result in 
neither satisfaction nor dissatisfaction regardless of whether they 
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have been fulfilled. Color-fastness was rated as an Indifferent 
requirement for the stripping basket. 

R Reverse CXialitv Elements : Requirements which result in 

dissatisfaction even when fulfilled or in satisfaction even when not 
fulfilled. (This usually indicates a problem in the question.) 

Q Questionable Result : May result from an error in the data- 

gathering process, or from an erroneous assumption about what is a 
functioning (or dysfunctioning) question. 

The relationship between the above three critical categories and the 
satisfaction or dissatisfaction experienced by the customer is expressed 
graphically below. Indifferent elements are represented by the x-axis. 



Satisfaction 



Dysfunctioning 




water drainage 
corrosion 
resistance, 
neutral color 
line shifts 
natural size loops 
ease of adjustment 
conformity to body 
light weight 
outer edge droop 
interference with 
movement 
sun resistance 
line fallout 
position stability 



It is important to note that the axes in the above diagram are 
asymmetrical, that dissatisfaction is not the opposite of satisfaction. 
When an Attractive requirement goes unfulfilled, the result is not 
dissatisfaction, but simply a lack of satisfaction. In contrast, fulfilling 
a Must-be requirement does not produce satisfaction. It only avoids 
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dissatisfaction. Only one class of requirements behaves as if the 
positive and negative axes were continuous: One-dimensional factors. 
These requirements can induce reactions ranging from dissatisfaction, 
through indifference, to satisfaction, depending on how well they are 
fulfilled. 

The Kano question pairs will help us to locate each requirement on the 
above graph. When pieced together, the answers to a 
functioning/dysfunctioning question pair identify which of the five 
categories a given customer need fits into. The process of merging the 
information contained in these response pairs is the purpose of the Kano 
evaluation table described in the section, "Processing Results." As an 
example, though, a response to the functioning question of "I like it that 
way" generally indicates that the requirement fits somewhere in the 
right side of the above diagram, along the satisfaction axis. The 
response to the dysfunctioning question would pin down the requirement 
as either Attractive or One-dimensional. 

The quality element curves are not necessarily stable over time. The 
curves can shift down and to the right over time as the market becomes 
conditioned to expect certain features. Thus, a feature once considered 
Attractive, such as the remote control for a television set, could move 
over time into the One-dimensional category, and ultimately become a 
Must-be requirement. This phenomenon is termed "quality satisfaction 
decay" by Professor Shop Shiba. 

In developing and testing the questionnaire, you became familiar with 
the five standard responses to each of the question pairs (ranging from 
"I like it that way" to "I dislike it that way"), and may be curious 
about the order in which they appear. The logic behind the 
arrangement of the resp>onses is the level of pleasure experienced by the 
customer. A scale of pleasure is known as a hedonic scale. 

The question is frequently posed: why is "I like it that way" a stronger 
statement of pleasure than "It must be that way?" Consider these 
responses in ^e context of Kano's functioning question. The thought 
behind this ordering is that the first res{X)nse signifies a type of 
positive satisfaction, while the latter relates to avoidance of 
displeasure. Additional investigation of the hedonic scales is currently 
in progress. 
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Continuous/Graphical Analysis 



There are two px)werful advantages to using continuous variables: 

• A continuous approach can summarize the data without losing 
resolution. For example, in the Kano evaluation table there are 
nine response pairs which equate to the "Indifferent” 
dimension,and each may have a somewhat different emphasis. 

• A continuous representation deals more comfortably with situations 
where there is no dominant response to a question (e.g., 37% Must- 
Be, 33% Attractive, 30% One-Dimensional) by allowing for 
intermediate p>oints, or hybrids. See the caveat below, however. 

Caveat : The continuous analytical approach described here is best 
applied after inspecting responses for evidence of discrete market 
segments. Lumping together distinctly different perspectives to form an 
average may produce confusing results. Take, for example, a case where 
votes are evenly split between "Attractive," "Must-Be," and "One- 
Dimensional." One way to check for the existence of distinct segments 
is to run a test of correlation between this response variable and some of 
the demographic data collected with the questionnaire. When 
different segments are identified, we suggest handling the data 
separately. 



To solve the problem of data loss, we can assign each response a 
numerical value, establishing a scale. We propose the following levels 
for the functioning and dysfunctioning responses: 



Functioning 

Like=4 
Must be=2 
Neutral=0 
Live with=-l 
Dislike=-2 



Dysfunctioning 

Dislike=4 
Live with=2 
Neutral=0 
Must be=-l 
Like=-2 



Each of the Kano dimensions may now be represented as a coordinate 
pair: 



X Y 



Reverse -2 

Indifferent 0 

One-dim. 4 

Must-Be 4 

Attractive 0 



-2 

0 

4 

0 

4 
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Functionality 



These cx)ordinates will serve as reference p>oints for the other data 
(when averages are taken across large data sets, rarely will there be 
"pure" Must-be or Attractive requirements). Answers generally lie 
somewhere in between. With this continuous representation, we can 
retain that information. 

The logic for the asymmetrical scale (beginning from -2, rather than -4) 
is that Must-be and One-dimensional dimensions are stronger resp>onses 
than Reverse, or (Questionable. Therefore, our scaling should give less 
weight to such responses to diminish their influence on the average. 

The following map should help to clarify the positioning of the various 
dimensions. You may want to comp>are it to the Kano evaluation table 
presented earlier. 



Kano Response Map 



Like 

(+4) 



Must-Be 

(+2) 



Neutral 

( 0 ) 



Live with 
(- 1 ) 



Dislike - 
(- 2 ) 



Q 




A 


A 


A 




0 













R 

R 

R 

R 



Like 

(- 2 ) 



Indifferent 

(I) 



R 



R 



R 



M 



M 



M 



Must-Be Neutral Live with Dislike 
(-1) (0) (+2) (+4) 

Dysfimctionality 
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For each Kano question, compute the average of the functional responses 
(these will be mapped on the y-axis) and the average of the 
dysfunctional responses (to be mapped on the x-axis). With a properly 
designed survey, the averages should fall in a range of 0-4, since 
negative values (which indicate questionable or reverse dimensions) 
should be in the minority. 

We can now form a grid (shown opposite), with Xave and Yave ranging 
from 0 to 4 on each axis. Note that each comer of the grid represents a 
prototype, a pure result. For instance, if all res|X)ndents to a given 
survey question answered "like" to the functional portion and "dislike" 
to the dysfunctional piece, the coordinates of the average response 
would be (X=-h 1, Y=+4), indicating a pure (Dne-dimensional requirement. 

Notice that the square is divided into quadrants. When the pairs of 
coordinates representing the average responses to each of the Kano 
questions have been plotted on the grid, the nature of each requirement 
is clearly delineated by the quadrant into which that point falls. For 
instance, a requirement such as number 5, which falls into the up|:>er left 
quadrant, should be viewed as an Attractive element. 

The closer a point falls to one of the four labeled comers (the 
prototypes), the more unaninwus the survey respondents must have been 
in their views. Conversely, a point such as number 9, which falls near 
the center of the diagram, is a fuzzier result which indicates 
disagreement among respondents. 

Shading each point according to the average importance vote for that 
requirement is an easy way to integrate the information obtained from 
the Self Stated Importance questionnaire. 

Interpretation 

There is no hard-and-fast way to interpret the diagram for 
development priorities. The best approach might vary with the 
number of points falling into each quadrant, with the clustering of the 
points within a quadrant, or with the degree of differentiation of 
importance levels within a quadrant. For instance, in the above 
diagram, there is only one Attractive element. Although it ranks only 
as medium in importance, the team might believe that the product 
needs a differentiating characteristic. 

What makes the most sense is for your group to view the grid (perhaps 
without the points numbered, in order to maintain objectivity about 
which requirements should be pursued), and then agree on a decision 
rule that will work for your data. 
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Functionality 



Average Functionality vs. Average Dysfunctionality 
(Coded by Average Importance, and with Questions Numbered) 



Attractive I 






1 One-dim 








!• 




59 


90 


30 ~ 




29 






— 


8C 


lOo 


4® O 7 

6® 


Indifferent I 






I Must-be 



Dysfunctionality 



Average 

Importance 



V 



3.0- 3.9 O 

4.0- 4.9 0 

5.0- 5.9 m 

6.0- 6.9 ® 

7.0- 7.9 • 
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Pouibl« enhancoments 

• Use the importance votes to weight the Kano responses. Using this 
method means that the more importance a respondent ascribes to a 
given requirement, the greater his or her influence will be in 
classifying the requirement as One-dimensional, Must-Be, etc.. 
Remember to use the weighted version of standard deviation if you 
want to include error bars on this graph. 

• Where resjxjnses to a particular question have been grouped into 
distinct market segments, plot all the points on one graph, but in 
different colors. If the graph becon>es too cluttered, you may have 
to omit the information on variation or importance. To avoid losing 
the importance data, you might vary the color only on the number 
next to each point. 
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Appendix E: Concept 

Engineering 
Process Metrics 



The metrics subcommittee of the CQM Research Committee has been 
investigating product development process metrics. This appendix is 
contained in the Autumn '92 issue of the Center for Quality 
Management Journal. 
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Appendix F: Concept 
Engineering 
Worksheets 

Customer Requirement Worksheet F.2 

Metric Development Worksheet F.3 

Kano Questionnaire Worksheet F.4 

Self-Stated Importance Worksheet F.5 

PM I Worksheet F.6 
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Ctistomer vnine Image Kev Item Requirement Statement 
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Metric Development Worksheet 



Customer Requirement! 



Ambiguity: 




Possibilities 




Everyone 


A difference 


for multiple 


Everyone 


interprets 


in interpretation 


interpretations 


agrees on 


differently 


exists 

1 1 1 


exist 

1 1 


meaning 

1 


1 


1 1 1 

2 3 4 


1 1 

5 6 


1 

7 



Metrics 


Validity 


Feasibilit 

Y 


Rank 











Kano Questionnaire Worksheet 
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la. 


1. I like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. 1 can live with it that way. 
5. I dislike it 


lb. 


1. I like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it 



2a. 


1. I like it that way. 

2. It must be that way. 

3. 1 am neutraL 

4. I can live with it that way. 

5. I dislike it 


2b. 


1. I like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it 



3a. 


1. I like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it 


3b. 


1 . 1 like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it 



4a. 


1. 1 like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it 


4b. 


1. I like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it 



5a. 


1. 1 like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it 


5b. 


1 . I like it that way. 

2. It must be that way. 

3. 1 am neutral. 

4. I can live with it that way. 

5. I dislike it 
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Self-Stated Importance Rating Worksheet: 



I. How important is it or would it be if: 



Not at All Somewhat Very Extremely 

Important Important Important Important Important 





1 


2 


3 


4 


5 


6 


7 


8 


9 


2. How important is it or would it be if: 


1 


2 


3 


4 


5 


6 


7 


8 


9 


3. How important is it or would it be if: 


1 


2 


3 


4 


5 


6 


7 


8 


9 


4. How important is it or would it be if: 


1 


2 


3 


4 


5 


6 


7 


8 


9 


5. How important is it or would it be if: 


1 


2 


3 


4 


5 


6 


7 


8 


9 


6. How iTnportant is it or would it be if: 


1 


2 


3 


4 


5 


6 


7 


8 


9 


7. How important is it or would it be if: 


1 


2 


3 


4 


5 


6 


7 


8 


9 


8. How important is it or would it be if: 


1 


2 


3 


4 


5 


6 


7 


8 


9 


9. How important is it or would it be if: 


1 


2 


3 


4 


5 


6 


7 


8 


9 


20. How important is it or would it be if: 


1 


2 


3 


4 


5 


6 


7 


8 


9 
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P.IUS - M.lnus - interesting Worksheet 



Concept Description: 


P 




M 




1 
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MEETING ANNOUNCEMENT 



COM 

The Center for Quality Management 
150 CambridgePark Drive, Cambridge, MA 02140 
Tel#; (617) 873-2152 Fax#: (617) 873-2155 



Please deliver a copy of this fax to each person in your company at the receiving 

fax number listed below. 

TO: CE User's Group 



NAME: 


COMPANY: 


PHONE#: 


FAX #: 


.-.Ian K. Grahan 


Center for Quality Management 


Hm (508) 369-8714 


CQM ( 


617) 873-: 


,'c.nn H. Petrc-i.ni 


Teraoy.ne, Inc. 


(617) 


422-2216 


(617) 


422-2910 


r.ane Shen 


Bolt Beranetc and Newman Inc. 


(617) 


873-3730 


(617) 


873-5011 


:ary Burcnill 


MIT 


(617) 


258-5586 


(617) 


258-7579 


'\i;resh PariK.h 


Cigita^ Equipment Corporation 


l508) 


493-1551 


(508) 


493-6094 


-a.ph P. Anders wT. 


STU I-nternat lonal 


(508) 


667-4111 


(508) 


667-9068 


’eg Doyle 


Praxis International Inc. 


(617) 


661-9790 


(617) 


497-1072 


Ace Kasacula 


Polaroid Corporation 


(617) 


577-5056 


(617) 


577-4022 


David Boger 


Bose Corporation 


(508) 


879-7330 


(508) 


820-4865 


H'jgh Loveday 


Ford Motor Company 


(313) 


322-0886 


(313) 


322-4033 


Christina Brodie 


Polaroid Corporation 






(617) 


577-2882 


Bill Fetterman 


Analog Devices, Inc. 






(617) 


937-2000 


Dawn Dougherty-r itcgerald 


MIT 






(617) 


253-5771 


Pan Chan 


EPA 












MIT 


(617) 


253-2229 


(617) 


253-5771 


Mike Timko 


.Analog Devices, Inc. 


(617) 


937-1257 


(617) 


461-4496 


Blise Locker 


Bolt Beranek and Newman Inc. 


(617) 


873-6327 






Cr.ristopher Mccre 


Bose Corporation 


(508) 


879-7330 


(508) 


872-6541 



FROM: Ted Walls PAGE #: 1 DATE: 3/5/93 

RE: Monthly Meeting 

The CE User's Group minutes and Meeting Announcement are 
attached, with addenda from the meeting. 
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COM MINUTES 

150 CambridgePark Drive, CE User's Group 

Cambridge, MA 02140 

Tel #: (617) 873-2152 Fax #: (617) 873-2155 Feb 19, 1993 at Bose Corp, Framingham 



In attendance: Ralph Anderson, BTU; Mike Timko, Analog Devices; Elise Locker, 
BBN; Christopher Moon, Bose; Yogesh Parikh, DEC; Jay Thomas, Sippican; Joe 
Kasabula, Polaroid; Gerry Waldron, BTU; Marty Soderlund, BTU; Diane Shen, 
BBN; Gary Burchill, MIT, Mark V. Martin, MIT; Hugh Loveday, Ford; Dawn 
Doherty Fitzgerald, MIT; David Boger, Bose. 

Next Meeting 

Friday, March 19, 1993, from 3 to 6 PM, at Polaroid. The CE User's Group meets 
the third Friday of every month. 

Highlights 

1. Mike Timko. ( Analog Devices) presented a method for using results of the 
Kano and Self stated Importance Questionaires in Concept Selection. Mike has 
developed an Excel Spreadsheet which uses the "Attractive", "Must Have", One 
Dimensional", and "Indifferent" scores directly with the SSI values to generate the 
"better than", and "worse than" factors. (Eliminates the need to decide the Kano 
category.) 

2. David Boeer (Bose) (discussed the use of Concept Engineering Techniques to 
establish "Fitness for Use" during Alpha-Testing of a product. This structured 
method used voice of the customer interviewing of a number of people (12-20) 
who used an engineering sample of a product for a short period. (2 weeks?) 

CE methods were used to "explore" the responses, develop requirements, 
weight and prioritize the opportunities for improvement. 

This Alpha testingwould have been easier if the project had been started 
using CE techniques (SSI and Kano factors available). 

The reaction by the development team to the results was very favorable, 
with appreciation for the clarity of the list. 

3. The announcement for upcoming CE course was reviewed, with minor 
changes suggested. 
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The leaders of the day were were agreed upon as follows: 







Leaders 


Assistants 


Day 1 
Steps 1,2 


May 3 


John Petrolini (Teradyne) 
Diane Shen (BBN) 


State Siting Board 


Day 2 
Steps 3,4 


May 5 


Christina Brodie (Polaroid) 
Ralph Anderson (BTU/CQM) 


State Siting Board 


Day 3 


May 7 




Diane Shen (BBN) 
State Siting Board 


Steps 5,6,7 




David Boger (Bose) 


Sippican (J. Thomas) 


Day 4 


May 11 


Elise Locker (BBN) 


Sippican (J. Thomas) 


Steps 7,8 




Kennv Likis (BBN) 


..Praxis (P. Doyle) 


Day 5 


May 13 


Joe Kasabula (Polaroid) 


..Praxis (P.Doyle) 


Steps9, 10-15 




Mark Skilling or Bruce Amazeen 
(Analog) 



note: 1. Yogesh Parikh has offered to fill in where needed 
2. Diane Shen may get substitute for 1 Day. 

Leaders of the Day are requested to have copy ready masters of their material to 
CQM by April 14. 

4. Dawn Dougherty Fitzgerald and Mark Martin (MIT Leaders of Manufacturing 
Program) reviewed the feedback from the LOM Concept Engineering Course in 
January. (5 each of 1/2 day session) 

They abstracted the KJ results of the weaknesses-mainly insufficient time 
and inexperience of the instructor with teaching the material. 

Gary Burchill led an effort to capture suggestions for improvement (Labels 
collected by stage of CE) the upcoming course. 
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Concept Selection Spreadsheet 



< 
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Analon Dtjvires Inr 



Table 1 : Two Dimensional Table of Evaluation 











Dysfunctioning 










1 . like 


2. must- 

be 


3. neutral 1 

1 


4. live 
with 


5. dislike 


I 
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1 1- 

I 


like I 


Q 


, A I 


A ! 


A 

1 


0 


Func- 


2. 


must-be 


R 


1 

\ 


' 


1 i 


M 


tioning 


I 3. 

I 


neutral 

I 


R 


1 

( 


1 


I 1 i 


M 




4. 


live with 


R 


1 


1 1 


' 1 ' 


M 




, 5. 


dislike 

i 


R 


R 

1 


1 R 


, R 


i Q 



Kano Questionnaire Interpretation 

Satisfaction 
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My Kano Questionnaire Interpretation 




Better = 



A + 0 

A + M + O + l 



xSSI 



Worse = 



M + Q 

A + M + O + l 



xSSI 
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Kano Response Data 
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The Use °f Concept Engineering Techniques 
To Establish Fitness-for-Use For New 
Products 



CE Us^s Group Mating February T 9 , 1993 



Situation: 

Nearing the completion of new product 
development, prototypes are field-evaluated 
to test fitness-for-use 




CE Ustta Group Mooting Fobruory I9. 1993 
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step 1 : 

Distribute between 12 to 20 devices to 
evaluators 



Puroose 

• ExDose evaiuaior to oroauci-unaer-tesi in iheir home environment 

• Evaiuaior generates written notes regaraing operation 



CE User s Group Meeting February 19, 1993 



Apply results of Griffith & Hauser research 



How to choose evaluators? 

• von Hippie Lead User theory 
Evaluation Time 

• Acceleration of exposure to product-under-test 
Evaluator's Notetaking 

• Conscious recall of 80 oercent of material lost within 24 hours 



CE User s Group Meeting February i9, 1993 



Step 2: 
Exploration 



Purpose: 



Collect unfiltered information 



CE User' a Group Meeting Febnmry 19, 1993 



Simple Discussion Guide 



• What did you like the most? 

• What did you like the least? 

• If you were the designer, what would 
you change? 

Use of Kawakita’s Principles of Collecting 
Language Data 



CE Ussr s Group Mooting Fobruary 1 9 , 1993 




Step 3: 

Extract Weakness-Related Voices/Net Touch 



Purpose: 

• Focus on areas not fit for customer use 

• Organize data to streamline downstream processing 



CE Usar's Group Mooting Fobruary 19, 1993 




step 4: 

MPM on “must-be” issues 



Purpose: 

• Focus on “must-be*’ or “showstopper ” issues 
{use Kano s dimensions) 

• Develop a manageable set of issues 



CE UsoTM Group Mooting Fobruary 19, 1993 



step 5: 

Translation 



Purpose 

• i ransiaie rrcm evaiuator s language into a lorm on wmcn the 
Deveiooment Team can act 

Create unamoiguous and unrestnctive reouiremeni statements 



Step 6: 

Operationally Define Requirements 



PufDose 

* Obiea!veiv oeftne reauirements 

• Create system to assess pertormance aiiornaiives 




Write reauirements for each voice. Then MPM requirements, 
not voices. 

Use Kano survey to weight/prioritize requirements. 

On CE projects, assess evaluator requirements versus CRs. 



Assumptions 



Testing fitness-for-use employs ‘exploratory ’ research methods 
■ not • confirmatory ■ methods. 

In simplest form. Development Team is capable of dimensioning 
issues as articulated in “voice ” or raw data form. 
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Appendix C: Inductive System Diagrams 



The diagrams in Appendix C come from the Inductive System Diagram 
test conducted in the winter of 1992/1993. The first diagram in a series, i.e. 
Diagram #1.1, is the original diagram with the common variable names 
annotated. The second diagram, i.e. Diagram #1.2, is the revised diagram drawn 
with the common variables. 
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Inductive System Diagram Test (#2) 
Diagram # 1.1 




Inductive System Diagram Test (#2) 
Diagram #1.2 




lerstanding 



Inductive System Diagram Test (#2) 
Diagram # 2.1 
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Inductive System Diagram Test (#2) 
Diagram # 2.2 




287 




00 

00 

CN 



Inductive System Diagram Test (#2) 
Diagram # 3.2 




Inductive System Diagram Test (#2) Diagram #4.1 
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Inductive System Diagram Test (#2) 
Diagram # 4.2 




Inductive System Diagram Test (#2) 
Diagram # 5.1 
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Inductive System Diagram Test (#2) 
Diagram # 5.2 




Support for Shared 
Process Understanding 

not in diagram 



Inductive System Diagram Test (#2) 
Diagram # 6.1 
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Appendix D: Selected KJ Diagrams 



The KJ diagrams in Appendix D illustrate my approach to selective coding 
analysis for the variable Design Objective Clarity. A complete review of all field 
notes for each team was conducted and references which might be related to 
Design Objective Clarity were selected for consideration. As a result, all KJ data 
labels represent actual observations of, or statements by, product development 
team members. 

The selected KJs represent each of the basic units of the comparative 
analysis. Team 1 A came from company 1, used Concept Engineering, and 
exhibited time to MARKET orientation. Team 2A came from company 2, used 
Concept Engineering, and exhibited time to MARKET orientation. Team 2B 
came from company 2, did not use Concept Engineering, and exhibited TIME to 
market orientation. Team 3A came from company 3, used Concept Engineering, 
and exhibited TIME to market orientation. 

The final KJ diagram represents a synthesis of the individual team KJ 
diagrams. The original data labels for this KJ were the first level grouping titles 
from the individual team KJ diagrams. These first level titles were then grouped 
and a second level title was prepared. The data labels shown on the diagram in 
this appendix are actually the second level titles from this analysis. 
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What did I observe about design objective clarity from team lA? 



Concept development is not emphasized 

Concept development work 
is not recognized as valuable 



Design activities 
can change direction 






Thisprocea# 
would have 
provided a clearer 
vision, a strategK 
path to the end 
itself 

Stop the number of 
deadends. Stop 
paths with 
significant ri^t 
turns and left turns 






I just have this 
continuum of fear 
or pressure is that 
knowing while I’m 
doing this there is 
no visible output 



Concept 
development is 
not a priority 
activity 

/^ery difficult 
get saiTie sense o^ 
urgency from 
anyone for a 
product early in 
its gestation 



V: 



•Ihe 

development 
stage isn't the 
biggest thing on 
anyone's plate 



Product definitions done 
by others are suspect 



Analysis promotes confidence 



t 



Engineers did 
not consider 
the marketing 
function to be 
competent 

Most of the people 
I've worked with 
from marketing 1 
would fire 
immediately 

1 go to marketing 
and say what are 
the things these 
people are looking 
for. Then I go to 
marketing and say 
these are the things 
you haven't 
considered 



Product 
development 
specifications 
done by others 
not trusted 






The very fact 1 
had face-to-face 
interviews it 
certainly helps to 
have confidence 
we are on the 
right path 

How do you know 
the researcher 
wrote what the 
customer said 

His (marketing 
rep) worst 
ni^tmare is that 
a bunch of 
engineers are 









Documented analysis promotes 
management support 



Systematic analysis 
promotes confidence 



Documented 
audit trails 
reduce 
management 
concerns 



Senior 

executives can 
be convinced 
with 

documented 



analysis 



If the data is 
really good, and 
well-presented, 
the credibility is 
up and the 
attacks are 
down 

By documenting 
a trail perhaps 
you head off 
some of the 
restrictions that 
get imposed on 
you 






I have high 
confidence and 
better ability to 
sell to executive 
committee 
because the 
process is self- 
documenting 

Concept 

selectiOT process 
provides 
credibility by 
verifying 
intuitive concept 
to someone 
(PRB) who 
doesn't 
understand 
what is going on 






Confirmation of 
personal 
perspective 
with customer 
data increases 
confidence 



Fear is reduced 
when decisions 
can be 
supported 
confidently 



Validation of 
what I thou^t 
was right for the 
market is a rvet 
gain 

After the 
interviews the 
data is 
somewhat 
different 
although for the 
most piart I 
could have put 
down the 
parameters a 
couple of 
months ago. 
Now I have 
hi^r 
confidence 



/ W« 



We will have 
exhausted every 
arguement they 
(PRB) can bring 
before us I 
would no longer 
feel nervous 

If we follow a 
process which 
presents 
objective data, 
you don't have 
anything to hide 
from. You have 
enough data to 
support your 






Maybe I’m 
spoiled by this 
process because 
it has become 
intuitive to me. 
A3F solution led 
to A4F 



Confidence in results 



I am willing to 
sign on the 
dotted line as an 
internal spec 



This absolute 
belief was 
unshakable, no 
one could knock 
me off 



1 get the warm 
and fuzzy 
feeling 




’j selecting the ^ 

\ product specs 

\/ 



Contextual awareness 
improves understanding 



Customer contact 
clarifies requirements 



Requirement 
interpretation 
differences can be 
settled by 
reviewing 
customer context 



They began to 
discuss what a 
label meant and 
went back to the 
customer voice 
and rewrote the 
label 

Disexplaining a 
grouping. 
Others are 
explaining their 
versions. B 
wants to know 
the context of 



Customer 
exposure 
influences 
designer actions 

/• N 

f When you go ^ 

• and get toe-to- 

I toe with 

customer it is 
dramatically 
different than 

* being in the 

t plant 



Ail the stuff 
worked for me 
because 1 was 
continually 
feeling for the 
customer; every 
input modified 
what 1 was 
[ doing 

t 

I 1 would dearly 
I have loved to 
I have one of the 
I designers 
I involved to get 
I an appreciation 
I of why you re 
putting this here 
or that there to 



I » 
I ® 

V 



get exposure to j 
customer focus 
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Individuals determine design outcomes 



Designers must make tradeoffs 



Designers cannot do everything 



Designers are 
constrained by 
factors outside 
their control 



Designers want 
to focus on the 
vital few 
recjuirements 



/ 



However, can’t 
shoehorn a 
requirement into 
a boK that 
doesn't fit 

1 get constrair^ 
by materials 
selection I don’t 
have control 
over, and design 
constraints I 
don’t have 
control over 



Try to weed out 
as many 
possibilities as 1 
can early 

We can’t do 
everything so 
let’s foctis on the 
critical few 

B<ScB look at the 
remainiivg iabeb 
and state: 1*m 
not going to 
vote for 
anymore; I’d 
rather say 21 
than go 24 and 
have some 

J 



Designers want 
to know relative 
requirement 
importance 

1 have to know 
which tradeoff 
buys more than 
It loses 

That’s where i 
need the input 
that tells me 
where 1 need the 
\^rrphasis 



Desi 



Designers need to 
understand how 
factors interact 



;ignerswork^ 
backwards from 
an image of the 
finished product 

s, 

1 have to get the 
big picture first 
1 have to know 
that it feels right 
first 



Committed people 
make positive progress 



[ see the finished 
product and 
then I see myself 
using it and then 
I wnte the sp>ecs 



Features are so 
interrelated so I 
could not do it 
in a vacuum 






Process participation promotes understanding 



Discussions are 
used to clarify 
written statements 



There were 
several different 
discussions 
about what the 
real requirement 
was; eventiuUy 
there was a 
concensus of the 
requirements’ 
meaning 



During Image 
label final round 
MPM selectees 
discussed what 
they saw in the 
labels they select 



Someone that 
hasbiiyin 
understands the 
program, 
understands the 
requirements, 
understands the 
how and why, 
and can explain 
to other people 
horizontally or 
vertically 



Participation in 
the process 
builds 

understanding 

J This whole 
process, by 
going through, 
we have 
generated 
separately, we 
all converge on 
the solution 



Thispn 
allows us ail to 
see the data and 
gain buyin 

The outcorres of 
the events we 
have been 
through 
together, 
excluding the 
CRKJ were 
surprising. 

Now the results 
of the CRIQ are 
not surprising to 
me 



V 

V 



Commitment to 
design objectives 
promotes design 
realization 



r. 






Post program 
approved then 
run in 

automatic, hard 
work but people 
already know 
what to do and 
want to do it 



It is safe now. 
Once it is 
defined and 
concensus is 
built it will stay. 

I don’t have to 
worry that some 
mutant child 
will appear m its 
place 



Passionate 
participants 
ensure the work 
gets done 



They have 
ownership. 

They don’t want 
to let other 
people down 

1 think that 
where there is 
passion there is 
ownership and 
those two 
combined; when 
they exist in the 
same group of 
people and the 
team encounters 
problems and 
difficulties they 
don't last 

Every single day 
that passion 
enabled us to 
come in and 
fight the good 
fight to do the 
things that 
needed to be 
.done 



Individual bias can 
determine the design objectives 



Senior 

managers can 
dictate product 
attributes 

Tcan see ^ 

Japanese country 
manager coming 
up with a 
requirement 
which is 

completely out of 
phase 

The top 

executive threw a 
product idea out 
on the table and 
it was assumed 
as a mandate 



The chairman 
made a decision 
on the spot to 
include X 



J 



Personal 
agendas can be 
pushed during 
design activities 

WeVehad ^ 
people who lost a 
contract because 
they couldn't do 
a certain 
thing.. .Takes 
90% of emphasis, 
but the missing 
item may be only 
1% of customers 

We used to make 
it exactly the way 
we wnated 
before this 



When pushing 
the marketing 
rep for the 
customer voice 
he stated, T don't 
realty have a 
voice for that I | 
made it up I 
because 1 thmk j 
it’s irrportanL"^/ 



J 



March 5, 1993 
Bedford, MA 
Gary Burchill 
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What did I observe about design objective clarity from team 2A? 

Requirement statements cannot stand alone Understanding is based 

on preceding experiences 



Requirement statements needed clarification 



Placing 
requirement 
statements in 
context of 
original voice can 
clarify intent 



Metric 

development 
clarified the 
intent of some 
requirements 



CXiring metric 
development B 
goes back to a 
worksheet and 
reads the original 
voice 

We will come 
across so many 
deosaon 
points — self- 

allows US to go 
back to where and 
what customers 
said 



MS reads a 
label S makes 
a comcnenl MT 
says TMo, it 
means..." MB 
states what he 
thinks it means 



J 



/^During metric ^ 
* development the 
team 

conversations 
appear to be 
directed to 
clarifying 
technical 
characteristics of 
the requirement 

]; I am beginning to 
dhiTige tny mmd jh 
the ambiguity 
rating we gave this 
requirement When 
we kx>k at the 
metric It is not at all 
ambiguous what 
^we want to do v 






There are tight trade- 
offs, severe limits to 
what you can do. 
PrioritLZ^ those is 
mainly what you get 
from the customer 



Focus determines effort 

Limited focus concentrates effort 






Designers want to 
focus on the vital 
few 

/C last to select 
Finally he 
states,'’Nah, I don't 
want any of these* 
and walks away 
without selecting 
anything 

During the first 
round of CR MPM, 

MT states.**! like all 
of these but I have ^ 

\ to be selective j 



Discrete bytes of 
information are 
easier to process 

r 

f It seems easier to be ) 
able to capture 
bytes of 

information and 
carry that through 
the process rather 
than a huge story 
board 

Breaking it down 
into individual 
pieces it was 
important to me to 
separate every Idea 
on the paper that 
allowed us to focus 
^ reflection on each r 



Clear objectives 
direct designer 
efforts effectively 






Understanding 
design objectives 
clarifies what to 
work on 

A consequence of 
really 

understanding 
customer 
requirements is 
you know what is 
important and 
what is not 
important 

They wUl know 
what to solve and 
what not to solve 



PriontLaed 

requirements tell you 
what IS good 
enough, where you 
have to put efforts, 
or where you can 
cocrprocnise 

Designers won't 
get ori on a tangent 
spending a week 
or two chasing 
something they 
think b neat, they 
will be more 
effective 



Unclear design 
objectives 
reduce 
confidence 



{ ^ 

? A consequence of i 

unclear ob)ectives is 

you second guess 

the project during 

development 

A consequence of 
unclear objectives 
is you have a team 
J which questions 

J 



Product definition activities build on prior efforts 



Ideas are 
connected to the 
results of 
previous work 

MS Is reading his 
label to the group 
aivd walks over to 
the Image KJ and 
points out the 
connection to the 
Idea 

During idea 
generating B aits 
in front of the 
Quality Chart and 
alternates be tw een 
looking at the QC 
and looking at the 
par«l periodicalJy 
new tdcM^ 



By knowing the 
roots they are 
going to know 
how we got to 
where we are in 
the product 
definition 



Requirements 
were clearly 
linked to 
customer voices 

find the \ 
power to be able 
to link one voice 
to one image to a 
key item to a 
requirement 

When I use 
logical /systems ti 
c thinking m this 
I think here is 
what he said aivd 
here is the 
requirement 
which matches 
what he said 



Exposure builds support 



Senior 

management 

clearly 

demonstrated 
support for the 
team 



The CM starts the 
IK) session by 
telling the team his 
corporate annual 
hoshin dealt with 
creating value 
through VoC 

When it was 
mentioned that 
meat tickets were 
available it brought 
surprise and 
satisfaction from 
several team 
.members 



Process 

participation 

builds 

process 

confidence 

^ The people 
bitching about the 
time are not the 
people on the 
team 

When we got into 
the room there was 
built-in credibility 
because the person 
who is processing 
the informatian is 
witnessing data 
collection ^ 



J 



• wit 



The existence ^ 
of a common 
background 
eases conflict 
resolution 

**^^cy (mkting. & 
eng)areboth 
going to know 
where the 
definitions came 
from and that 
will make the 
resolution of 
conflict a lot 
easier 

Since we have gone 
through this 
process digesting 
requirements and 
looking at 
marketing 
perspective and we 
came to consensus, 
we can elimmate 
trivial arguments 
during design i 



Systematic analysis is thorough 



The team took time 
to ^ thorou g h 



MS states, *X3k. 
everyone happy 
with this?"; .MT says. 
**No, we need to 
think about tha one 
for a minute.* 

Some suggesticns 
from the team move 
on as the group 
appears to be 
addressed 
elsewhere. MS 
suggests they reileci 
on the group's 
strengths for new 
ideas. Sa ideas get 
V^enerated . 



During the check 
for omission, MT 
states "Ohhh, that 
one is not captured 
anywhere el^, put 
it up there 



v: 
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Systematic 
analysis reduces 
downstream 
sxirp rises 

We have covered so \ 
much it will be hard 
to think we haven't 
thought about 
something 

Once we decide 
what we are going 
to do there 
shouldn't be any 
surprises 

I sense a more 
direct path to end 
product — not 
missing things that 
will come back to . 
^h awtyouUte^ ^ 






r 



Awareness of use 
environment can influence design 

Actual experiences carry credibility 



Stories of real 
experiences are 
used to clarify 
points of 
discussion 

MT answers a ^ 
clarification 
question with a 
reference to an 
existing part and 
the problems 
with it 

An Idea comes 
up about second 
sourdng: MB 
tells a story 
about a 

competition, MS 
tells a story . 
^a tout a cu^ome ^ 



The secorul item 
of credibility b 
we have an 
identified group 
of people when 
or if push comes 
to shove we can 
go back to 



Requirements 
anchored in 
customer 
experiences have 
credibility 



Asa team 
member, it is 
credibility that 
the ideas from 
others were bom 
from a mstomcr 
need 

in aiwwer a senior 
manager question 
on performance 
spec the seruor 
engineer answers 
with “Some of the 
people we talked 
to... "senior 
manager nods his 
head 



Z/ 



Customers view 
the designed 
part in a broader 
context 



One level up 
perspective — not 
kx)kingat my 
piece but seeing 
hew my piece fits 
in 

Customers have 
their ovm 
priorities; my p>art 
is only one of 
many things the 
customer has to 
\^ ^orry about 



The factors that 
detenrone 
whether a 
product is 
successful are so 
different product 
topnoductyou 
have to adapt 



Designers put themselves first 

Designers typically make 
decisions from their vantage point 



Designers don't 
always consider 
the implications 
of their decisions 
on others 

-V 

It used to be our ^ 

goal was to just | 

meet the basic | 

sprees without 
worrying about 
what other p>eople 
do 

You make an 
arbitrary choice 
during design arvi 
fx.'d Out Uter it's \ 

much harder (for | 

other* to deal wrth) J 



We used to do 
product definition 
by the-seat-of-the- 
p>ants; Judge irdo 
from customer from 
your perspective 
which tends to be 
biased 



Designers typically 
discount inputs from 
non -tyhnical pe ople 

/^^get sales* inpu^ 
in design process 
we tend to 
invalidate them or 
see this as not as 
important as our 
techiucal irrsight 

Normally, you take 
a young marketer 
ai^ send him out 
aiul say go talk to 
customers and they 
come back and telJ 
the engineers what 
the customers want 
aiul the engineers 
say he doesn't kiKiw 
what he's talking 
Vbout j. 



Traditional development can lose 
sight of customer requirements 



Dominant 
individuals can 
dictate design 
objectives 

^ 

Last project we 
wort^ on we had 
authoritarian eng. 
manager who told 
us exactly what to 
do whether we 
wanted to or not 

The biggest 
dilemma in the 
past is that a few 
very strong 
characters defined 
the product 
whether it was fine 
or not 



During 

development, 

some 

requirements can 
be abandoned 



Traditioncil 
approach to 
capturing 
customer 
comments is 
inefficient 



^ All too often you ' 


^ 


put blinders on. 




^ Led me to believe ^ 


getting in a rut 




that trying to 


a ttacking one piece 




capture what 


of the problem and 




customer said with 


letting other asp^ects 




a one-liner every 


slide 




few minutes and 


We used to set the 




looking a week 


requirement 
(solution) and 




later, ridiculously 
inefficient 


then go off and try 




Out capture of the 


to do it If we 




(customers) 


could not do it we 




answers is 


would make 




normally 


decisions that it 




spxjradic and wee 


must not be that 




lose a lot of these 
\^wers 


inportant if we 





Designers find it 
difficult to 
change existing 
designs 



Once you design 
it, it's hard to 
change it, there are 
things you can do 
up front 

Typically a 
designer starts 
working and it's 
based on his or 
her ideas. Not 
until you do a 
review do you get 
other ideas; it's 
too late to make 
\ ^\anges then 
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What have I observes about design objective clarity from Team 2B? 

Customers had minimal influence on the design objective 



The design was influenced by factors 
other than customers 



The engineer was focused 
on technical considerations 






^Understand mg The engineer The engineer 

techrdcal viewed the wanted to 

trade-otfs performance begm the 

increased the specs as a by- design 

engineer's product not discussion with 

confidence objective of the technical issues 

***'T*l'Vit kMhail w»i« «• wfif u mm m»mm 



oooiplMtly daSriKl 
bt« 1 WM eorl«d«ni 
hovteoak^tt 



OmatMol 
oonad«KB kb< 

*bl* K> 4 m« • 



batwMn fouthand 



L (skulilloni M<d j 



kdltl(K>lgillk«w« 

tOlPBWt ttw 

OMapMtton LTb 
nMwm wtatfou 

wa.«l»«d<in'i«« 
babatwrihui thaf 







what «»• itHnfc It 

kiL_z 



E>esign objectives were 
influenced by environmental 
influences 



/Hargett 



t martlets Design 

were objectives are 

determined influenced by 

from reviewing corr^titive 

the corr^>anies products 

strategic market ^ ’■ " 
analysis 



/ At Aal lha 

BaAMtncnp I 



A/t— n ' H igtii nf tha 
ptMipratMta^f 



I 

yw 



Analt*iitoa<Saa4tn 

Wi daoda arhai typa 
of pnduci iff loekint 

pfoducM. what hol« 



Company employees, 
without the teams 
expiertence, can 
influence the design 



Other designers 
are a primary 
influence on 
the engineer 









Product 
attributes can 
be determined 
by people in 
power who 
Lack detailed 
understandin g 



M: Dofwt (tank 70V 
naad le pubHak • 
Uiih MDo; raec4a 



•M tMjtaa U* Qi 
L wMdt tia rfkocaiflf i 



The marketing function didn’t transfer customer insight effectively 



The marketing rep was reluctant 
t o share customer source data 

^he marketing The marketing"^ 
rep was reluctant rep did not 
to give the share his actual 

engineer access to notes from 
customer contacts customer visits 



yz 



M. nj ff»» 3>OM th* 



with the 
engineer 



DuAnfkdM 



MM WatJwuld 



K- So Ifon hM» do 
jmMwtr* 10 kook •« 
Con cm SCut 
wo|iM M« wtkch 

wtut/Uwohov* 
oa* Mildoirt 



Marketing reports contained 
less information than 
desired by the engineer 

Direct customer contact 
provides the engmeer 
more information than 
marketing reports 



'^TkSJnca 

)au4ikO(«« 



(oaooitlvai 

daiUtfouai 



•oy -fMA dM« ma b« 

glOOr MdMtllUA 






doraitnnr El 
klndolwvi* 

«vy*unyo»r»or»» 

Customer contact promotes 
engineer’s sense of connection 
with actual users 

^h»..oi»i 

«h»«»<dtpeliciBo« oao I •» foaf a c\»«o»a*yoM ••• 
70V kmw th* mum ^ andin« ««• a don « 

oltcaaMM who* »oiiai>»» w»vo wdJ kraw il ir* 10 . M. 

u-n, II b* avifw am nod » ***" < 1 







Early access to an actual design was important to the team 
The dsigne^vantedar^c^^ 



prior to discussions with others 






e engineer 
wanted an 
actual design in 
order to talk to 
customers 
about trade-offs 




An actual 
design was 
the basis for 
discussions 
with other 
designers 

You noodlob* w*f7 
^ukk a|«i 
tMMhaf on 



You Larntiound 
Ian (ooiBp«n7 2) 

wkii(ut»t 



The team wanted to start 
design activities quickly 

^e marketing rep The engineer^ 
pushed to shorten wanted to start 



the product 
definition time 



p«m. I ank a 

•«alywtta pwa 
ind lh«i • awk 



M. Alda f« 
iiaa«i«a« 

ohoiddboM 



design 

activities prior 
to having a 
corrplete 
understanding 




YoutiMlavoio 

jvkDpiAond 

laiieon 



The proposed design 
objective was not compelling 



The team 
acknowledged 
the decisions 
could change 
downstream 



The senior 
marvagement group 
was not committed to 
the proposed product 
definition 



E W>af<7ouM7 

indent* 



di»r 

atll danf»— yu*. 
hop«lullY nM OMCli 



•oaumi pioduci 



AiPUB. loO ^ 

Mkuot E mtrt a 

t^kppfOVll Ifll 
iMotEMya 

'* T anart aioufb 
ana lan.*Mh« 
wra ih* ipptovt^ 
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Design objectives were established 



^ Relative priorities are ^ 

In design you must make choices ^tablished during the^^en 

The designer can deade 



r 



You must make a 
design objective choice 



You must state what you are 
ro^ to do to arr*ar confid ent 



You hiv* «e »pp«M 
oorM«nL You anrtpt M7 
1 WD tNlSOrf ibotf <lo 
Hi*” rou ho-** «> mtf TMi 
to what t unSotni' 



To fat iMn|B dona, to 
biaatha Ida into It row 
iMwa to fool conf idan 
anoi^ to «y Wa tea 
fotnf todoir 



It is necessary to commit 
to a product definition 



f Tm ol ft la ayutf rtua la 
It hava to tona on 



loaia point wa haaa 
U dta dtoaion afcM 
iliypaolpfQducw 



9 



The product definition 
focuses on the key product 
r<^luirement 

( ItMiStboutdia AptodiKidtHwaonia \ 
alttJfawandnaka J-ia fm bty innbuiaa I 
ihaoitlwklll^taaa I 

< y 



Design requires 

maKins tradg-gffs 




Commitments were based 
on conservative estimates 
of ptfTformance 



"Playing it 
safe* leads to 
confidence 

< N 

rMolbaanf 



Trade-offs ^ 
are necessary 
in design 
activities 



down Ttwyu 



MShoiddtaaba 
looUnf lot a e o» 
plaoa a Aialhay 

•Klutfaal M: 
Wtatfwa can-1 
do both. wMdi la 






re lative requirement prio n ties 



The designer The engineer 
made the made a 
determinatio decision on 
what the 
customer's 
need-to- and 
nice-to-have 
reqvureiTYents 




diaf d Uw ta j 

V— 1>7 



Established priorities 
orient trade-off decisions 




Data deficiencies affected the decisions 

Decisions were made with data deficiencies 



Decisions were make with incomp4ete data 



The team 
recogruzed it 
was making 
decisions with 
incomplete data 



k/ww wnai w» an 

doaif. Uwa 
dia cn aai anyildnf 



daciuanaihayeaA 
on aaaiMbaa di<a 
ihwi fvtunf OMra 




Critical 
engineering 
concerns were 
not discussed tr 
the m ee ti n gs 



honaaity dotf I kiww 
T laaly didn't fw a 
faaiforfaai. 1 |<aI 



E; CAI w«d <d Anal 



Decisions were made with inappropriate data 



The team guessed at 
the values for some 
decision vanables 



M.U<aai 



thaaL Old yuu |W onW E 
No. I didn\ but d couM* 
rdbabam* M:AnyidaaJ 
EToiatya 
knawadid U 



nucr^Ma M.-'Haia 



fuMa mytw 10 w«^ oosta «v* 



The senior manager 
focused on inappropriate 
data from the decision 
under discussion 



Ai PAS k laanafavfoaa 

ID boaid and drawa frcpA 
K TMa wott^ halp m 



foatfratoU E la did 



Sr M«<. fo* • board w> 
dioMa C>*ph M; Thd am 
haip ua dactda how nai 
wa tfa foaif toaall Sa. 
Mfc. Nd II wdl aM Am 
4 pac Wa aia wofbutf m 
wTunf piaca of data 



Information availability affects the deasion process 

r A lack of data undermines 
the decision process 



The availability of data 
increases confidence 



A lack of data 
can create an 
impasse if 
opinions differ 



e rd bha «a «U «D 
paeplaiMi^ vn U: 
That la MM Aw Mr*. 



Than M aay* Aaa • 

InpoattM. Thaewa 
doanbwdcd Awarfy 
l<mnt And 






In the absence 
of data, people 
present 
opinions to 
justify design 
objectives 

M. r«a boon tryiftf 10 






of haw. but now lard 
|ot Biy opiMon ooiha 
woord. laoaa lot of 
C wt havoto (a*C 

U you aia not auw tha 
daw w nrhi you aay 1 
Aimfc wa naod dAa. that 
lawhaiyouwy 
baciM that W aduii 
Lroutttatb 






Personal data 
collection 
promoted data 
confidence 



r irhid 



•rhKM 
undaaaa 
(hay ata wlia>f 



e, PaacHha 1 apoetOc 
apac by wtarran lo II 
indaiad wa 



Data increases 
the confidence 
in ideas 



aiy old wy T ww 
and (woua^paaa 
apparwia. X awd C 
lwaa|otn« 
AifOw|ti lha *aa 
and llda aaa 
maadi wtwi vy 

TNa p«aa aa all 









The teatn interpreted some issues differently 

^pitawMa A difference in ^ 

terminology 
definitions existed 



r AI rxE Afaw Ml 

M: Sul Will dial ttw attaopk E ataampw w 
ddwawnaf e>iant«f anawara O-aocvncy 
E Thara ivm how | quwoen. wnior 
daf tna It a 

noiacruwcy * E 
a<cufa«y. pwowoiv tha 
MOW dune Odwaa; 

V Thay. 



by E who la In lum 



Odwaa; . 
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What did I learn about design objectives clarity 
from team 3 A? 



Individuals did not 
identify themselves 
with team results 



Process participation builds contextual connections 



Individuals had agendas 
unrelated to team efforts 

^ bidividuals 
tried to 
reduce their 



The team 
felt senior 



exposure 

wanted to 
put things 
down that 1 
couldn’t 
deliver, 
didn’t want 
to get backed 
into a comer 

People right 
now are doing 
anything to 
keep their 
jobs — includin 
g keeping 
their mouths 
shut, doing 
what 

manage rrtent 
says, whether 
it is right or 
wrong , 



R had 8 black 
Ubeb 
completed. 

Everyone else dictate 
was grouping, solutions 
L (noticing) 

"How can you 
be writing 



managers 

would 



without 
reference to 



blues without t eams effo rt 
looldxtg at the 



groupr 



If we get ' 
Tony 

involved he 
will want 
things certain 
ways, that 
whole %veek 
of CE stuff 
wiUbe 
wasted. 

If you throw 
up solutions 
now artd get a 
relativeJy 
strong 

response, we 
could end up 
designing the 
system vt that 
roocTC and 
they are not 
isjxi the team! > 



Agreement to move on did not 
mean agreement with results 






Agreement to proceed 
mean agreement with 
decision 

'' ^ 

You get a ccwnmittee ’ 

solution again. It's the least 
common 

denominator... people don’t 
see anything wrong, but 
they are not partictilarly 
happy with It either 

Discussion of this first 
group took 20 minutes. 

People were discussing 
conibinations with 
labels outside the 
group, and labels were 
shuffled around the 
board to create 
hypothetical groupings. 

No one seemed 
particuLarly satisfied 
with group or title 
^when they moved on J 



does not 
prior 



W complains 
he cannot 
think so buit 
He thinks the 
pace 

threatens 

"reprodudbil 

ity- 



Image KJ is 
easier because 
no emotional 
attachment as 
inCRK) 



Common understanding 
facilitates ^oup interaction s 
/Common ^ 



experiences 
build better 
reiaiigflships 

^ttiink the 
advantages, 
despite the 
continuity and 
involvement 
are having a 
group of people 
who share 
some common 
experience 

If develop- 
mentand 
mrkt hear 
more of the 
same stuff 
from 

customers it 
has to build a 
better 
working 

yrelatior^p ^ 



Common 

perspectives 

diminish 

au’guments 

^^hat 1 think %vc^ 
need to do is get 
us ail in a room 
and discuss 
them. Things 
that don’t make 
any sense, we 
would agree feo 
letga 

1 kind of saw it 
as we spent a 
lot of time on 
the up front end 
part. I though 
we would have 
less things to 
argue about 
later , 



Process participation provides more 
information than process output 
documentation 

^ Documenta 



tion lacks 
the 

affective 
qualities of 
interactions 



Sometimes 1 
wonder about 
this. When 
you hear 
something 
from the 
customer it 



Turnover 

created 

inconsistent 

levels of 

background 

knowledge 



r It isdocu- 1 


makes sense. 


l^ith turnover we T 


mented, but It 


If you extract 


will have 


b still not the 


it to the 


backwards 


same; it b not 


yellow stickie. 


movement 


like bringing 


it’s a problem 


because we will 


one team 
member on 




have big 
difficulties in 


We have lost 




getting consistent 


some people 




thinking on the 


who did good 




team 


vbits. We 
have lost what 




New people will 
not be as 


they heard, 
what the 




grounded in the 
customer 


customer did, 
all we have b 
y their notes ) 




requirement 
l^background } 



Management did not support the team 



i 



The scope of the project was not 
clearly established at the beginning 



We fluctiuted as to 
whether we wanted 
to develop a whole 
product or just a 
package 



Also there was 
confusion originally 
whether we were 
talking about Aries 
charts or the entire 
Aries system 



Team members assignments were disruptive 



Interview 
teams had 
skill 

deficiencies 

C the 4 core 
team members 
who did 

interviews did no 
shallow water 
swims 

The core team 
came up with a 
list of 

questions we 
were to ask on 
our visits. My 
personal 
experience was 
that I didn't 
use it 

My first clue 
was when the 
interviewer 
handed me a 
set of his noted 
because he 
wasn’t sure die 
note taker was 
taking good 
lotes 



Assignments 
to the team 
were made a t 
the last 
minute 

2 of 6 core team 
members who 
did crunch work 
did no 
interviews 

Won’t really 
know until the 
15th OKI) as to 
who is on the 
core team 
It was almost a 
fluke that I 
was there at 
the training. I 
was a 

substitute at 
the Last 

minute. 1 was 
not clear when 
1 went what 
my role was 



The team 
experienced 
massive 
turnover 

^Everya^gineer^ 
signed to the 
team either 
transferred, quit, 
or was 6red 

5 <rf the 6 people 
participating in 
stages Z 3, and 4 
did not 
participate in 



The team perceived a lack of 
management commitment 



us, gaining a 
better 

understanding of 
the customer's 
problem is 
important, but to 



At this point it is 
real clear other 
companies have 
much more 
management 
commitment than 



y ^them (PRB) so what we have 

Management actions 
created delays 

Engineering 



Management 
personnel were created a delay 
committed to of several 

other projecte months 

/nierewasadeUy 



This didn't get 
the attention 
particularly of 
engineers 
because they are 
tied up on other 
projects 

If 1 have to take 
engineers out 
(VoC visits) we 
are so resource 
constrained it 
means skipping a 



Gan to May) in 
the interview 
process 

It is mainly the 
mixed message 
of thinking we 
had a clear 
guidance and 
then being told 
to stop 
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Customer's context clarifies requirements 






Abstracted statements can create 
divergent interpretations 

A statement carT^ 
be interpreted in 
different ways by 
d ifferent peopl e 

Tasked, "What iT^ 
guidance? An on- 
line hetp 
functionr They 
replied that it 
was not, that it 
was information 
on how to 
analyze data, sort 
of an expert 
system 
A discussion 
begins. This 
requirement has 
somewhat 
different meaning 
to the participants 

He says, "I 
understand the 
words, but I 
don't 

understand 
what they are 
saying" A 
discussion arises 
to further refine 
.the title 



er levels of 
abstraction 
reduce darity 
of original 
intent 

i^don't think we 
can get it to a 
or question. 

It's too hi^ a 
level of 
abstraction 

The higher we go, 
the more we deal 
with semantic 
probiema, and we 
redefine the 
things that 
already have 
meaning. You 
can't change the 

y Oxibffd dictionary^ 

A CR substantially 
rewritten during 
CRK) Ubel 
scrubbing was a 
source of 
frustration 
(vagtjeness) in 
metric and Kano 
development 



Requirements anchored in context are clearer 

Customer 
statements 



without Very often the 

^ ^ discussions 

context 

decreases semantics 

requirement probfems, not 

^derstandin g 

/Cv ^ weren't clear 

When you re 

looking at 
images of VoC, 
and getting 
conclusions in 
requirements is 
the only methcxi 
to clearly define 
a product 

When you do 
VoC and the 
customer says, 

"1 like X," if you 
don't 

investigate, you 
don't know 
what the 
^message means 



References to 

customer 

context resolved 

debates 

regarding 

requirement 

understanding/ 

meaning 

^KeferwKwto 
experience in 
customer's 
environment was 
used to clarify 
requirement and 
develop metric 
Some ccxifusion 
arose over a 
label Some 
participants 
offered their 
interpretation. L 
located and 
reread the 
context this 
settled the matter 
and discussion 
moved to the 

S. next label 



JJ 



The customer's starting point is 
a very concrete way that a thing 
should happen. Ours is always 
a very general method 



Individuals had an understanding of 





Objective clarity focuses efforts 



Clear requirements direct efforts 

N 

Requirement clarity 

promotes focused Defined criteria 
dev dopment effo rt promote focused 
progress 



Knowing the 
requirements you 
can make a real 
product 
specification 

If you create a 
new product 
it's efficient 
because the 
first solution 
you get is 

«,^timized a 



When you don't 
have rules, you 
can go in the 
wrong direction 

I'd have a hard 
time eliminating 
more (ideas) 
than one or two 
without dear 
criteria 



Customer 
empathy builds 
customer 



/Vk 



By getting mto 
the customer's 
mind, they 
understand the 
product s utility 
Getting oriented to 
customer needs. In 
some cases 
changing biases 
and getting 
intimately familiar 
with customer 
requirements and 
environments 



1 



February 27-28, 1993 
Bedford, MA 
Gary Burchill 
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What have I observed about design objective clarity? 




Concept 
development 
work is not 



gjven priority 
by management 



Concept 
development 
activities are not 
always supported 
by management 

Credible 
engineering 
personr^ were 
not assigned to 
oneCE team 




Some design decisions are hard to justify 

PN . . . Some design objectives are 

Decisions to move on are ? ^ ^ ^ 

j not important to customers 

s ometimes made ]ust to move o n ^ — 

^ f Influential 

Agreements to The 
proceed are 



sometimes 
make simply 
to move on 

N 

R^gardlesa, 

design 

objectives are 
established 

Some decisions 
arc not made 
with conviction 



development 
team is 
pressured to 
s how progre^ 

There is 
pressure for the 
team to produce 
results 

Development 
activities can 
start prior to 
establishing clear 
^^d«ign 



analisysis 
not always 
relevant 

^^ere are 
opportunities to 
lose customer 
requirement 
fidelity 

Design objectives 
were established 
with recognized 

k data deficieivries 



Personal 
agendas can 
become design 
jectives 



o^ 

f?t\ 



Personal 
agendas can 
influence design 
objectives 

The engineer 
was focused on 
terimicaql 
considerations 

People in 
positions of 
power can 
dictate product 
design 
^.^ives yj 




Designers make choices during design 
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Understanding decisions increases confidence 




\ 



Context increases understanding 




Disagreements on 
design objectives can exist 



Unsupported Requirement 

inferences (opinions) statements can be 
create disagreements interpreted 
differently 



A Uckof dati can 
create an impasse if 
opinions differ 

Marketing and 
engineering typically 
don't trust each other 



Distance from context 
increases requirement 
misinterpretatian 
opportunities 

Statements can be 

interpreted 

differently 




V. 



Supportable 

decisions build confidence 



Supporting 
evidence builds 
management 
committment 



The ability 
to support 
decisions 
increases 
confidence 



^viderKe ' 




f The ability to \ 
support a 


supporting 




decision 


design 




increases 


objective 




confidence 


influences 




Individuals 


management 




wanted to 


committment 




make 


Documented 




conservative 


analysis 




committments 


increases 




Systematic 


management 




analysis 


V committment > 




builds 




V. cordidence J 



Decision process 
understanding 
influences 
confidence in 
decision outcomes 




process can build 
confidence in the 
process results 



Requirements 
linked to 
customers have 



credibility 

Discrete bytes of 
information are 
easier toproecss 



J 
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